[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:07:07 dist-test-slave-chkf krb5kdc[832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:07:07 dist-test-slave-chkf krb5kdc[832](info): set up 2 sockets
Sep 25 13:07:07 dist-test-slave-chkf krb5kdc[832](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260925 13:07:08.305346 825 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:07:08 dist-test-slave-chkf krb5kdc[832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:07:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:07:08.419032 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:45893
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:45893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:08.497582 845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:08.497821 845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:08.497866 845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:08.499019 845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:08.499085 845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:08.499109 845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:08.499137 845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:08.499166 845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:07:08.500602 845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:45893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:45893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:08.500882 845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:08.501183 845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:08.506307 851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:08.507951 850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:08.507951 845 server_base.cc:1034] running on GCE node
W20260925 13:07:08.508224 853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:08.508518 845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:08.508872 845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:08.510041 845 hybrid_clock.cc:648] HybridClock initialized: now 1790341628510022 us; error 32 us; skew 500 ppm
Sep 25 13:07:08 dist-test-slave-chkf krb5kdc[832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:08.512593 845 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:07:08.513098 845 webserver.cc:460] Webserver started at http://127.0.206.126:32989/ using document root <none> and password file <none>
I20260925 13:07:08.513360 845 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:08.513405 845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:08.513500 845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:08.514532 845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "e146ecce9dbe4f87867e7a3409a05682"
format_stamp: "Formatted at 2026-09-25 13:07:08 on dist-test-slave-chkf"
server_key: "9abe59a59bb68031c9f3d8d407acfe8d"
server_key_iv: "b47e53e2a5ca2697d587e8f110d70860"
server_key_version: "encryptionkey@0"
I20260925 13:07:08.514914 845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "e146ecce9dbe4f87867e7a3409a05682"
format_stamp: "Formatted at 2026-09-25 13:07:08 on dist-test-slave-chkf"
server_key: "9abe59a59bb68031c9f3d8d407acfe8d"
server_key_iv: "b47e53e2a5ca2697d587e8f110d70860"
server_key_version: "encryptionkey@0"
I20260925 13:07:08.516319 845 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:07:08.517030 860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:08.517225 845 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:07:08.517300 845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "e146ecce9dbe4f87867e7a3409a05682"
format_stamp: "Formatted at 2026-09-25 13:07:08 on dist-test-slave-chkf"
server_key: "9abe59a59bb68031c9f3d8d407acfe8d"
server_key_iv: "b47e53e2a5ca2697d587e8f110d70860"
server_key_version: "encryptionkey@0"
I20260925 13:07:08.517393 845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:08.550043 845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:08.551862 845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:08.552030 845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:08.555617 845 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:45893
I20260925 13:07:08.555718 912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:45893 every 8 connection(s)
I20260925 13:07:08.556037 845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:08.556912 913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:08.559252 913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682: Bootstrap starting.
I20260925 13:07:08.559827 913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:08.560128 913 log.cc:824] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:08.560874 913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682: No bootstrap required, opened a new log
I20260925 13:07:08.562408 913 raft_consensus.cc:348] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e146ecce9dbe4f87867e7a3409a05682" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45893 } }
I20260925 13:07:08.562573 913 raft_consensus.cc:374] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:08.562620 913 raft_consensus.cc:729] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e146ecce9dbe4f87867e7a3409a05682, State: Initialized, Role: FOLLOWER
I20260925 13:07:08.562856 913 consensus_queue.cc:260] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e146ecce9dbe4f87867e7a3409a05682" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45893 } }
I20260925 13:07:08.562953 913 raft_consensus.cc:388] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:08.562997 913 raft_consensus.cc:482] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:08.563051 913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:08.563565 913 raft_consensus.cc:504] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e146ecce9dbe4f87867e7a3409a05682" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45893 } }
I20260925 13:07:08.563679 913 leader_election.cc:302] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e146ecce9dbe4f87867e7a3409a05682; no voters:
I20260925 13:07:08.563892 913 leader_election.cc:288] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:07:08.563921 916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:08.564090 916 raft_consensus.cc:686] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682 [term 1 LEADER]: Becoming Leader. State: Replica: e146ecce9dbe4f87867e7a3409a05682, State: Running, Role: LEADER
I20260925 13:07:08.564141 913 sys_catalog.cc:564] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:08.564119 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 845
I20260925 13:07:08.564174 916 consensus_queue.cc:237] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e146ecce9dbe4f87867e7a3409a05682" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45893 } }
I20260925 13:07:08.564218 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:07:08.564574 917 sys_catalog.cc:455] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e146ecce9dbe4f87867e7a3409a05682" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e146ecce9dbe4f87867e7a3409a05682" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45893 } } }
I20260925 13:07:08.564673 917 sys_catalog.cc:458] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:08.564844 918 sys_catalog.cc:455] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e146ecce9dbe4f87867e7a3409a05682. Latest consensus state: current_term: 1 leader_uuid: "e146ecce9dbe4f87867e7a3409a05682" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e146ecce9dbe4f87867e7a3409a05682" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45893 } } }
I20260925 13:07:08.564918 918 sys_catalog.cc:458] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:08.564872 924 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:08.565033 825 external_mini_cluster.cc:1428] Setting key b094738fb19caa1be3d9f2fe2d86d4a7
I20260925 13:07:08.565510 924 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:07:08.567126 924 catalog_manager.cc:1347] Generated new cluster ID: 59b7e90f17434f7490c7eb5ff4a50023
I20260925 13:07:08.567219 924 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 13:07:08 dist-test-slave-chkf krb5kdc[832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:08.580020 935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:08.566134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51764 (local address 127.0.206.126:45893)
0925 13:07:08.568444 (+ 2310us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:08.568451 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:08.568472 (+ 21us) server_negotiation.cc:407] Connection header received
0925 13:07:08.568611 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:08.568616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:08.568654 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:08.568784 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:08.570258 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:08.571398 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:08.572461 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:08.572859 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:08.572978 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:08.576408 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:08.576418 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:08.576418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:08.576428 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:08.578105 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:08.578554 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:08.578557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:08.578557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:08.578644 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:08.578941 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:08.578942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:08.578943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:08.579167 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:08.579342 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:08.579604 (+ 262us) server_negotiation.cc:299] Negotiation successful
0925 13:07:08.579754 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2259,"thread_start_us":66,"threads_started":1}
I20260925 13:07:08.594614 924 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:07:08.595178 924 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:08.600988 924 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e146ecce9dbe4f87867e7a3409a05682: Generated new TSK 0
I20260925 13:07:08.601174 924 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260925 13:07:08.633560 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45893
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:08.717020 939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:08.717208 939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:08.717231 939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:08.718446 939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:08.718487 939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:08.718513 939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:07:08.720007 939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45893
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:08.720249 939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:08.720474 939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:08.726670 947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:08.726734 939 server_base.cc:1034] running on GCE node
W20260925 13:07:08.726891 945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:08.727077 944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:08.727312 939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:08.727563 939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:08.728698 939 hybrid_clock.cc:648] HybridClock initialized: now 1790341628728678 us; error 39 us; skew 500 ppm
Sep 25 13:07:08 dist-test-slave-chkf krb5kdc[832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:08.731055 939 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:07:08.731498 939 webserver.cc:460] Webserver started at http://127.0.206.65:33805/ using document root <none> and password file <none>
I20260925 13:07:08.731758 939 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:08.731802 939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:08.731954 939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:08.732980 939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "98fcd9a20edc458ab68da5589b0a143b"
format_stamp: "Formatted at 2026-09-25 13:07:08 on dist-test-slave-chkf"
server_key: "50c3ecf1dd0772cfe8af77554441f0f2"
server_key_iv: "e7629bfb17efa92eed955359a24c115b"
server_key_version: "encryptionkey@0"
I20260925 13:07:08.733325 939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "98fcd9a20edc458ab68da5589b0a143b"
format_stamp: "Formatted at 2026-09-25 13:07:08 on dist-test-slave-chkf"
server_key: "50c3ecf1dd0772cfe8af77554441f0f2"
server_key_iv: "e7629bfb17efa92eed955359a24c115b"
server_key_version: "encryptionkey@0"
I20260925 13:07:08.734638 939 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 13:07:08.735333 954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:08.735496 939 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:08.735590 939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "98fcd9a20edc458ab68da5589b0a143b"
format_stamp: "Formatted at 2026-09-25 13:07:08 on dist-test-slave-chkf"
server_key: "50c3ecf1dd0772cfe8af77554441f0f2"
server_key_iv: "e7629bfb17efa92eed955359a24c115b"
server_key_version: "encryptionkey@0"
I20260925 13:07:08.735667 939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:08.756349 939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:08.758143 939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:08.758301 939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:08.758523 939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:08.758869 939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:08.758921 939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:08.758955 939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:08.758984 939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:08.764535 939 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:40571
I20260925 13:07:08.764590 1067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:40571 every 8 connection(s)
I20260925 13:07:08.764907 939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:07:08.768532 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 939
I20260925 13:07:08.768602 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:07:08.768757 825 external_mini_cluster.cc:1428] Setting key 7ae9c6dbf72d58e5c2855d7f6e6bdad8
Sep 25 13:07:08 dist-test-slave-chkf krb5kdc[832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:08.774667 935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:08.765699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:54881 (local address 127.0.206.126:45893)
0925 13:07:08.765767 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:08.765769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:08.765975 (+ 206us) server_negotiation.cc:407] Connection header received
0925 13:07:08.766424 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:08.766425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:08.766448 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:08.766502 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:08.767384 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:08.767916 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:08.769067 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:08.769333 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:08.769400 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:08.771574 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:08.771576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:08.771577 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:08.771585 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:08.773275 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:08.773684 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:08.773686 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:08.773686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:08.773747 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:08.774046 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:08.774047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:08.774047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:08.774145 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:08.774196 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:08.774537 (+ 341us) server_negotiation.cc:299] Negotiation successful
0925 13:07:08.774603 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 13:07:08.774994 1070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:08.765694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45893 (local address 127.0.206.65:54881)
0925 13:07:08.765851 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:08.765891 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:08.766343 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:08.766557 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:08.766561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:08.767114 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:08.767303 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:08.767307 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:08.767998 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:08.767998 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:08.768985 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:08.768988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:08.769419 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:08.769420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:08.769568 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:08.770091 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:08.770098 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:08.771486 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:08.773362 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:08.773367 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:08.773368 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:08.773592 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:08.773816 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:08.773817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:08.773818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:08.773977 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:08.774212 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:08.774218 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:08.774390 (+ 172us) client_negotiation.cc:769] Sending connection context
0925 13:07:08.774510 (+ 120us) client_negotiation.cc:240] Negotiation successful
0925 13:07:08.774688 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":92,"threads_started":1}
I20260925 13:07:08.775564 1068 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45893
I20260925 13:07:08.775686 1068 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:08.775846 1068 heartbeater.cc:511] Master 127.0.206.126:45893 requested a full tablet report, sending...
I20260925 13:07:08.776324 877 ts_manager.cc:194] Registered new tserver with Master: 98fcd9a20edc458ab68da5589b0a143b (127.0.206.65:40571)
I20260925 13:07:08.776896 877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:54881
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260925 13:07:08.793696 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45893
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:08.873811 1073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:08.874032 1073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:08.874055 1073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:08.875172 1073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:08.875211 1073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:08.875236 1073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:07:08.876582 1073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45893
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:08.876752 1073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:08.876919 1073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:08.881775 1081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:08.881767 1079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:08.881807 1078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:08.881981 1073 server_base.cc:1034] running on GCE node
I20260925 13:07:08.882211 1073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:08.882450 1073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:08.883611 1073 hybrid_clock.cc:648] HybridClock initialized: now 1790341628883610 us; error 32 us; skew 500 ppm
Sep 25 13:07:08 dist-test-slave-chkf krb5kdc[832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:08.885953 1073 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:07:08.886369 1073 webserver.cc:460] Webserver started at http://127.0.206.66:39967/ using document root <none> and password file <none>
I20260925 13:07:08.886543 1073 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:08.886582 1073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:08.886667 1073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:08.887672 1073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "eb7c878b563144d6ab37cfb5ffd2a593"
format_stamp: "Formatted at 2026-09-25 13:07:08 on dist-test-slave-chkf"
server_key: "914a533647ffaa3cd592457252577b31"
server_key_iv: "3bd8937db7f6b97cbf5d58260ae6ea6d"
server_key_version: "encryptionkey@0"
I20260925 13:07:08.888000 1073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "eb7c878b563144d6ab37cfb5ffd2a593"
format_stamp: "Formatted at 2026-09-25 13:07:08 on dist-test-slave-chkf"
server_key: "914a533647ffaa3cd592457252577b31"
server_key_iv: "3bd8937db7f6b97cbf5d58260ae6ea6d"
server_key_version: "encryptionkey@0"
I20260925 13:07:08.889178 1073 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:07:08.890008 1088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:08.890231 1073 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:08.890316 1073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "eb7c878b563144d6ab37cfb5ffd2a593"
format_stamp: "Formatted at 2026-09-25 13:07:08 on dist-test-slave-chkf"
server_key: "914a533647ffaa3cd592457252577b31"
server_key_iv: "3bd8937db7f6b97cbf5d58260ae6ea6d"
server_key_version: "encryptionkey@0"
I20260925 13:07:08.890386 1073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:08.900513 1073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:08.902222 1073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:08.902330 1073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:08.902588 1073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:08.902899 1073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:08.902951 1073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:08.902971 1073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:08.902984 1073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:08.908883 1073 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:35567
I20260925 13:07:08.908952 1201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:35567 every 8 connection(s)
I20260925 13:07:08.909250 1073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
Sep 25 13:07:08 dist-test-slave-chkf krb5kdc[832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:08.918192 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 1073
I20260925 13:07:08.918313 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:07:08.918334 935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:08.910179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:34775 (local address 127.0.206.126:45893)
0925 13:07:08.910224 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:08.910226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:08.910433 (+ 207us) server_negotiation.cc:407] Connection header received
0925 13:07:08.910999 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:08.911000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:08.911013 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:08.911075 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:08.911962 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:08.912468 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:08.913437 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:08.913649 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:08.913738 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:08.915793 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:08.915794 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:08.915794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:08.915802 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:08.917007 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:08.917408 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:08.917409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:08.917410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:08.917470 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:08.917707 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:08.917708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:08.917708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:08.917828 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:08.917904 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:08.918158 (+ 254us) server_negotiation.cc:299] Negotiation successful
0925 13:07:08.918243 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 13:07:08.918510 825 external_mini_cluster.cc:1428] Setting key bb60791c6dd58016ffb86f58787d511b
I20260925 13:07:08.918581 1204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:08.910165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45893 (local address 127.0.206.66:34775)
0925 13:07:08.910316 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:08.910355 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:08.910897 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:08.911147 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:08.911151 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:08.911689 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:08.911867 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:08.911871 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:08.912557 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:08.912558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:08.913347 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:08.913349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:08.913754 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:08.913755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:08.913873 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:08.914380 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:08.914388 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:08.915702 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:08.917105 (+ 1403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:08.917111 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:08.917113 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:08.917331 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:08.917537 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:08.917538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:08.917538 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:08.917639 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:08.917894 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:08.917899 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:08.918033 (+ 134us) client_negotiation.cc:769] Sending connection context
0925 13:07:08.918140 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 13:07:08.918291 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":78,"threads_started":1}
I20260925 13:07:08.919118 1202 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45893
I20260925 13:07:08.919232 1202 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:08.919443 1202 heartbeater.cc:511] Master 127.0.206.126:45893 requested a full tablet report, sending...
I20260925 13:07:08.919801 877 ts_manager.cc:194] Registered new tserver with Master: eb7c878b563144d6ab37cfb5ffd2a593 (127.0.206.66:35567)
I20260925 13:07:08.920315 877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:34775
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260925 13:07:08.942014 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45893
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:09.023571 1207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:09.023770 1207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:09.023797 1207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:09.024854 1207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:09.024886 1207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:09.024912 1207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:07:09.026252 1207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45893
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:09.026428 1207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:09.026594 1207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:09.031312 1212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:09.031301 1213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:09.031383 1207 server_base.cc:1034] running on GCE node
W20260925 13:07:09.031287 1215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:09.031737 1207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:09.031925 1207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:09.033066 1207 hybrid_clock.cc:648] HybridClock initialized: now 1790341629033057 us; error 28 us; skew 500 ppm
Sep 25 13:07:09 dist-test-slave-chkf krb5kdc[832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341629, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:09.035343 1207 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:07:09.035745 1207 webserver.cc:460] Webserver started at http://127.0.206.67:43553/ using document root <none> and password file <none>
I20260925 13:07:09.035920 1207 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:09.035962 1207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:09.036057 1207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:09.037014 1207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "7923c97db92f42348a798a7ebb18bc92"
format_stamp: "Formatted at 2026-09-25 13:07:09 on dist-test-slave-chkf"
server_key: "cc6019d646ebeb6e7bb48aef35bbacf6"
server_key_iv: "eefdb07207eaf6be7a667afb0ea75ea3"
server_key_version: "encryptionkey@0"
I20260925 13:07:09.037295 1207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "7923c97db92f42348a798a7ebb18bc92"
format_stamp: "Formatted at 2026-09-25 13:07:09 on dist-test-slave-chkf"
server_key: "cc6019d646ebeb6e7bb48aef35bbacf6"
server_key_iv: "eefdb07207eaf6be7a667afb0ea75ea3"
server_key_version: "encryptionkey@0"
I20260925 13:07:09.038478 1207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:07:09.039172 1222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:09.039328 1207 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:09.039443 1207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "7923c97db92f42348a798a7ebb18bc92"
format_stamp: "Formatted at 2026-09-25 13:07:09 on dist-test-slave-chkf"
server_key: "cc6019d646ebeb6e7bb48aef35bbacf6"
server_key_iv: "eefdb07207eaf6be7a667afb0ea75ea3"
server_key_version: "encryptionkey@0"
I20260925 13:07:09.039525 1207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:09.053951 1207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:09.055823 1207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:09.055974 1207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:09.056182 1207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:09.056475 1207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:09.056529 1207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:09.056571 1207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:09.056597 1207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:09.062608 1207 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:42141
I20260925 13:07:09.062646 1335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:42141 every 8 connection(s)
I20260925 13:07:09.062948 1207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:07:09.066438 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 1207
I20260925 13:07:09.066530 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:07:09.066726 825 external_mini_cluster.cc:1428] Setting key e64a33fc6cc1c144519ea0c51f9186dc
Sep 25 13:07:09 dist-test-slave-chkf krb5kdc[832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341629, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:09.072449 935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:09.063643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:55393 (local address 127.0.206.126:45893)
0925 13:07:09.063682 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:09.063684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:09.063860 (+ 176us) server_negotiation.cc:407] Connection header received
0925 13:07:09.064467 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:09.064469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:09.064485 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:09.064553 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:09.065612 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:09.066133 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:09.067497 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:09.067686 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:09.067779 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:09.069849 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:09.069851 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:09.069851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:09.069858 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:09.071100 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:09.071523 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:09.071524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:09.071524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:09.071583 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:09.071825 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:09.071826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:09.071826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:09.071939 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:09.072012 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:09.072278 (+ 266us) server_negotiation.cc:299] Negotiation successful
0925 13:07:09.072360 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:07:09.072645 1338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:09.063584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45893 (local address 127.0.206.67:55393)
0925 13:07:09.063739 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:09.063782 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:09.064356 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:09.064625 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:09.064629 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:09.065325 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:09.065516 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:09.065521 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:09.066227 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:09.066228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:09.067377 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:09.067380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:09.067788 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:09.067789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:09.067917 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:09.068429 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:09.068447 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:09.069756 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:09.071209 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:09.071214 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:09.071216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:09.071423 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:09.071648 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:09.071649 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:09.071649 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:09.071756 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:09.072016 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:09.072021 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:09.072150 (+ 129us) client_negotiation.cc:769] Sending connection context
0925 13:07:09.072257 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 13:07:09.072399 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":101,"threads_started":1}
I20260925 13:07:09.073122 1336 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45893
I20260925 13:07:09.073194 1336 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:09.073320 1336 heartbeater.cc:511] Master 127.0.206.126:45893 requested a full tablet report, sending...
I20260925 13:07:09.073642 877 ts_manager.cc:194] Registered new tserver with Master: 7923c97db92f42348a798a7ebb18bc92 (127.0.206.67:42141)
I20260925 13:07:09.074018 877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:55393
I20260925 13:07:09.078668 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 13:07:09 dist-test-slave-chkf krb5kdc[832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341629, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 13:07:09 dist-test-slave-chkf krb5kdc[832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341629, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260925 13:07:09.101296 1343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:09.093862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33598 (local address 127.0.206.65:40571)
0925 13:07:09.094033 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:09.094040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:09.094054 (+ 14us) server_negotiation.cc:407] Connection header received
0925 13:07:09.094085 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:09.094089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:09.094110 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:09.094195 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:09.094801 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:09.095519 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:09.096271 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:09.096593 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:09.096673 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:09.098846 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:09.098852 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:09.098853 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:09.098859 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:09.100243 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:09.100551 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:09.100553 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:09.100553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:09.100610 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:09.100786 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:09.100786 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:09.100787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:09.100907 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:09.101100 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:09.101154 (+ 54us) server_negotiation.cc:299] Negotiation successful
0925 13:07:09.101219 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":111,"threads_started":1}
W20260925 13:07:09.101648 982 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33598
Sep 25 13:07:09 dist-test-slave-chkf krb5kdc[832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 13:07:09 dist-test-slave-chkf krb5kdc[832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260925 13:07:09.122941 1343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:09.116263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33604 (local address 127.0.206.65:40571)
0925 13:07:09.116350 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:09.116353 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:09.116414 (+ 61us) server_negotiation.cc:407] Connection header received
0925 13:07:09.116512 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:09.116513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:09.116524 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:09.116595 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:09.117152 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:09.117645 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:09.118378 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:09.118588 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:09.118665 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:09.120802 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:09.120803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:09.120804 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:09.120812 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:09.121950 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:09.122281 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:09.122282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:09.122282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:09.122331 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:09.122487 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:09.122487 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:09.122488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:09.122575 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:09.122661 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:09.122804 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 13:07:09.122870 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 13:07:09.123500 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 939
I20260925 13:07:09.128351 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 1073
I20260925 13:07:09.133680 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 1207
I20260925 13:07:09.137995 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 845
2026-09-25T13:07:09Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1895 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:07:09 dist-test-slave-chkf krb5kdc[1350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:07:09 dist-test-slave-chkf krb5kdc[1350](info): set up 2 sockets
Sep 25 13:07:09 dist-test-slave-chkf krb5kdc[1350](info): commencing operation
krb5kdc: starting...
W20260925 13:07:10.158252 825 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:07:10 dist-test-slave-chkf krb5kdc[1350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:07:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:07:10.270213 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:42133
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:42133
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:10.348693 1363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:10.348917 1363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:10.348951 1363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:10.350070 1363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:10.350133 1363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:10.350158 1363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:10.350185 1363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:10.350208 1363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:07:10.351663 1363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:42133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:42133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:10.351918 1363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:10.352130 1363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:10.356789 1369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:10.356837 1363 server_base.cc:1034] running on GCE node
W20260925 13:07:10.356779 1371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:10.356781 1368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:10.357197 1363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:10.357544 1363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:10.358737 1363 hybrid_clock.cc:648] HybridClock initialized: now 1790341630358676 us; error 56 us; skew 500 ppm
Sep 25 13:07:10 dist-test-slave-chkf krb5kdc[1350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341630, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:10.361021 1363 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:07:10.361532 1363 webserver.cc:460] Webserver started at http://127.0.206.126:34977/ using document root <none> and password file <none>
I20260925 13:07:10.361754 1363 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:10.361819 1363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:10.361949 1363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:10.363029 1363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "f4a4913ace884c8bb198920b3435b57a"
format_stamp: "Formatted at 2026-09-25 13:07:10 on dist-test-slave-chkf"
server_key: "3be0cff7b39f6a394f66abfdae8bf793"
server_key_iv: "45a24d80c272ca1f7b3315e236c9f27e"
server_key_version: "encryptionkey@0"
I20260925 13:07:10.363365 1363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "f4a4913ace884c8bb198920b3435b57a"
format_stamp: "Formatted at 2026-09-25 13:07:10 on dist-test-slave-chkf"
server_key: "3be0cff7b39f6a394f66abfdae8bf793"
server_key_iv: "45a24d80c272ca1f7b3315e236c9f27e"
server_key_version: "encryptionkey@0"
I20260925 13:07:10.364661 1363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:07:10.365321 1378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:10.365489 1363 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:10.365582 1363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "f4a4913ace884c8bb198920b3435b57a"
format_stamp: "Formatted at 2026-09-25 13:07:10 on dist-test-slave-chkf"
server_key: "3be0cff7b39f6a394f66abfdae8bf793"
server_key_iv: "45a24d80c272ca1f7b3315e236c9f27e"
server_key_version: "encryptionkey@0"
I20260925 13:07:10.365660 1363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:10.380571 1363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:10.382292 1363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:10.382445 1363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:10.386389 1363 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:42133
I20260925 13:07:10.386420 1430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:42133 every 8 connection(s)
I20260925 13:07:10.386861 1363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:10.387687 1431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:10.389863 1431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a: Bootstrap starting.
I20260925 13:07:10.390385 1431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:10.390612 1431 log.cc:824] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:10.391315 1431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a: No bootstrap required, opened a new log
I20260925 13:07:10.392807 1431 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a4913ace884c8bb198920b3435b57a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42133 } }
I20260925 13:07:10.392935 1431 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:10.392957 1431 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4a4913ace884c8bb198920b3435b57a, State: Initialized, Role: FOLLOWER
I20260925 13:07:10.393096 1431 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a4913ace884c8bb198920b3435b57a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42133 } }
I20260925 13:07:10.393172 1431 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:10.393194 1431 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:10.393220 1431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:10.393664 1431 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a4913ace884c8bb198920b3435b57a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42133 } }
I20260925 13:07:10.393739 1431 leader_election.cc:302] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4a4913ace884c8bb198920b3435b57a; no voters:
I20260925 13:07:10.393873 1431 leader_election.cc:288] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:07:10.393945 1434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:10.394116 1431 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:10.394289 1434 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a [term 1 LEADER]: Becoming Leader. State: Replica: f4a4913ace884c8bb198920b3435b57a, State: Running, Role: LEADER
I20260925 13:07:10.394371 1434 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a4913ace884c8bb198920b3435b57a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42133 } }
I20260925 13:07:10.394640 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 1363
I20260925 13:07:10.394757 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:07:10.394757 1436 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4a4913ace884c8bb198920b3435b57a. Latest consensus state: current_term: 1 leader_uuid: "f4a4913ace884c8bb198920b3435b57a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a4913ace884c8bb198920b3435b57a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42133 } } }
I20260925 13:07:10.394835 1436 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:10.394950 825 external_mini_cluster.cc:1428] Setting key 11cae5dd99b54013654c81d784a1ddb9
I20260925 13:07:10.395062 1443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:10.395542 1443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:07:10.395969 1435 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4a4913ace884c8bb198920b3435b57a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4a4913ace884c8bb198920b3435b57a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42133 } } }
I20260925 13:07:10.396710 1435 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:10.397571 1443 catalog_manager.cc:1347] Generated new cluster ID: cc00010ba3be428c82cc8727836aa13c
I20260925 13:07:10.397624 1443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 13:07:10 dist-test-slave-chkf krb5kdc[1350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:10.405769 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:10.395542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48380 (local address 127.0.206.126:42133)
0925 13:07:10.396265 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:10.396274 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:10.396306 (+ 32us) server_negotiation.cc:407] Connection header received
0925 13:07:10.396421 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:10.396428 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:10.396482 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:10.396619 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:10.397377 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.398392 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.399202 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.399633 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.399756 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:10.401997 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:10.402001 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:10.402001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:10.402013 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:10.404294 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:10.404677 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:10.404680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:10.404681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:10.404769 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:10.404969 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:10.404969 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:10.404970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:10.405201 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:10.405359 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:10.405466 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 13:07:10.405603 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":184,"threads_started":1}
I20260925 13:07:10.432484 1443 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:07:10.433058 1443 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:10.438773 1443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4a4913ace884c8bb198920b3435b57a: Generated new TSK 0
I20260925 13:07:10.438972 1443 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260925 13:07:10.467464 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:42133
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:10.553081 1457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:10.553359 1457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:10.553393 1457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:10.554493 1457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:10.554527 1457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:10.554554 1457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:07:10.555949 1457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:42133
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:10.556116 1457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:10.556273 1457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:07:10.561100 1457 server_base.cc:1034] running on GCE node
W20260925 13:07:10.562470 1465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:10.562819 1462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:10.563043 1463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:10.563279 1457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:10.563520 1457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:10.564673 1457 hybrid_clock.cc:648] HybridClock initialized: now 1790341630564659 us; error 30 us; skew 500 ppm
Sep 25 13:07:10 dist-test-slave-chkf krb5kdc[1350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341630, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:10.567183 1457 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:07:10.567544 1457 webserver.cc:460] Webserver started at http://127.0.206.65:42113/ using document root <none> and password file <none>
I20260925 13:07:10.567710 1457 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:10.567747 1457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:10.567844 1457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:10.568869 1457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "ad1aec33a1da4fee9b4e1eac53ae6930"
format_stamp: "Formatted at 2026-09-25 13:07:10 on dist-test-slave-chkf"
server_key: "d7013d174de895352c19ee5fc0ce55f4"
server_key_iv: "d73af0ffd84297c243123204d264af9e"
server_key_version: "encryptionkey@0"
I20260925 13:07:10.569154 1457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "ad1aec33a1da4fee9b4e1eac53ae6930"
format_stamp: "Formatted at 2026-09-25 13:07:10 on dist-test-slave-chkf"
server_key: "d7013d174de895352c19ee5fc0ce55f4"
server_key_iv: "d73af0ffd84297c243123204d264af9e"
server_key_version: "encryptionkey@0"
I20260925 13:07:10.570452 1457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:07:10.571127 1472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:10.571288 1457 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:10.571372 1457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "ad1aec33a1da4fee9b4e1eac53ae6930"
format_stamp: "Formatted at 2026-09-25 13:07:10 on dist-test-slave-chkf"
server_key: "d7013d174de895352c19ee5fc0ce55f4"
server_key_iv: "d73af0ffd84297c243123204d264af9e"
server_key_version: "encryptionkey@0"
I20260925 13:07:10.571461 1457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:10.588047 1457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:10.589756 1457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:10.589916 1457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:10.590122 1457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:10.590416 1457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:10.590471 1457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:10.590510 1457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:10.590534 1457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:10.596534 1457 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:36509
I20260925 13:07:10.596598 1585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:36509 every 8 connection(s)
I20260925 13:07:10.596894 1457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
Sep 25 13:07:10 dist-test-slave-chkf krb5kdc[1350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341630, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:10.602555 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 1457
I20260925 13:07:10.602641 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:07:10.602838 825 external_mini_cluster.cc:1428] Setting key fd2b173d67c2bf1f0633c475eae47fde
I20260925 13:07:10.605717 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:10.597567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:40111 (local address 127.0.206.126:42133)
0925 13:07:10.597612 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:10.597613 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:10.597803 (+ 190us) server_negotiation.cc:407] Connection header received
0925 13:07:10.598213 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:10.598214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:10.598223 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:10.598268 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:10.599170 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.599672 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.600626 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.600817 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.600864 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:10.602863 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:10.602865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:10.602865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:10.602887 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:10.604289 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:10.604778 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:10.604779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:10.604780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:10.604841 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:10.605091 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:10.605092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:10.605093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:10.605205 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:10.605276 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:10.605540 (+ 264us) server_negotiation.cc:299] Negotiation successful
0925 13:07:10.605628 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 13:07:10.605878 1588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:10.597546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:42133 (local address 127.0.206.65:40111)
0925 13:07:10.597694 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:10.597725 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:10.598138 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:10.598323 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:10.598326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:10.598929 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:10.599099 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:10.599104 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.599775 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.599776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:10.600554 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:10.600556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.600883 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.600884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:10.600974 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:10.601371 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:10.601379 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:10.602760 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:10.604370 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:10.604376 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:10.604378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:10.604676 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:10.604908 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:10.604909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:10.604917 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:10.605023 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:10.605278 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:10.605284 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:10.605410 (+ 126us) client_negotiation.cc:769] Sending connection context
0925 13:07:10.605518 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 13:07:10.605661 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260925 13:07:10.606521 1586 heartbeater.cc:347] Connected to a master server at 127.0.206.126:42133
I20260925 13:07:10.606621 1586 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:10.606813 1586 heartbeater.cc:511] Master 127.0.206.126:42133 requested a full tablet report, sending...
I20260925 13:07:10.607285 1395 ts_manager.cc:194] Registered new tserver with Master: ad1aec33a1da4fee9b4e1eac53ae6930 (127.0.206.65:36509)
I20260925 13:07:10.607887 1395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:40111
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260925 13:07:10.625662 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:42133
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:10.703105 1591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:10.703330 1591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:10.703373 1591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:10.704567 1591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:10.704632 1591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:10.704679 1591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:07:10.706054 1591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:42133
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:10.706292 1591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:10.706508 1591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:10.711340 1597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:10.711314 1599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:10.711381 1591 server_base.cc:1034] running on GCE node
W20260925 13:07:10.711330 1596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:10.711742 1591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:10.711956 1591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:10.713120 1591 hybrid_clock.cc:648] HybridClock initialized: now 1790341630713110 us; error 24 us; skew 500 ppm
Sep 25 13:07:10 dist-test-slave-chkf krb5kdc[1350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341630, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:10.715569 1591 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:07:10.715969 1591 webserver.cc:460] Webserver started at http://127.0.206.66:40669/ using document root <none> and password file <none>
I20260925 13:07:10.716128 1591 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:10.716164 1591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:10.716248 1591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:10.717126 1591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "56833387211f414aa1790ab2abff49f7"
format_stamp: "Formatted at 2026-09-25 13:07:10 on dist-test-slave-chkf"
server_key: "1fe64370eb49b7043641a52692a95998"
server_key_iv: "077d30f16b0fe31c77d3d365c9db400f"
server_key_version: "encryptionkey@0"
I20260925 13:07:10.717439 1591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "56833387211f414aa1790ab2abff49f7"
format_stamp: "Formatted at 2026-09-25 13:07:10 on dist-test-slave-chkf"
server_key: "1fe64370eb49b7043641a52692a95998"
server_key_iv: "077d30f16b0fe31c77d3d365c9db400f"
server_key_version: "encryptionkey@0"
I20260925 13:07:10.718600 1591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:07:10.719347 1606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:10.719499 1591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:10.719576 1591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "56833387211f414aa1790ab2abff49f7"
format_stamp: "Formatted at 2026-09-25 13:07:10 on dist-test-slave-chkf"
server_key: "1fe64370eb49b7043641a52692a95998"
server_key_iv: "077d30f16b0fe31c77d3d365c9db400f"
server_key_version: "encryptionkey@0"
I20260925 13:07:10.719625 1591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:10.738039 1591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:10.739830 1591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:10.739934 1591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:10.740095 1591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:10.740314 1591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:10.740342 1591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:10.740363 1591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:10.740375 1591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:10.745615 1591 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:45291
I20260925 13:07:10.745766 1719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:45291 every 8 connection(s)
I20260925 13:07:10.745973 1591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:07:10.749615 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 1591
I20260925 13:07:10.749716 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:07:10.749871 825 external_mini_cluster.cc:1428] Setting key 35cc695ac1639d2e1c6b8f0cb88373b2
Sep 25 13:07:10 dist-test-slave-chkf krb5kdc[1350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341630, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:10.755091 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:10.746669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:51817 (local address 127.0.206.126:42133)
0925 13:07:10.746839 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:10.746841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:10.746963 (+ 122us) server_negotiation.cc:407] Connection header received
0925 13:07:10.747389 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:10.747390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:10.747400 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:10.747459 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:10.748311 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.748786 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.749742 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.750021 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.750096 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:10.752247 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:10.752249 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:10.752249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:10.752258 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:10.753601 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:10.754077 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:10.754078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:10.754079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:10.754136 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:10.754399 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:10.754400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:10.754401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:10.754516 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:10.754593 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:10.754914 (+ 321us) server_negotiation.cc:299] Negotiation successful
0925 13:07:10.755005 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260925 13:07:10.755270 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:10.746624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:42133 (local address 127.0.206.66:51817)
0925 13:07:10.746830 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:10.746869 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:10.747311 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:10.747511 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:10.747514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:10.748060 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:10.748239 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:10.748243 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.748867 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.748868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:10.749631 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:10.749633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.750135 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.750136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:10.750280 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:10.750852 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:10.750860 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:10.752160 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:10.753705 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:10.753712 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:10.753714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:10.753999 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:10.754204 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:10.754205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:10.754205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:10.754327 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:10.754586 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:10.754592 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:10.754772 (+ 180us) client_negotiation.cc:769] Sending connection context
0925 13:07:10.754893 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 13:07:10.755050 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":123,"threads_started":1}
I20260925 13:07:10.755704 1720 heartbeater.cc:347] Connected to a master server at 127.0.206.126:42133
I20260925 13:07:10.755788 1720 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:10.755976 1720 heartbeater.cc:511] Master 127.0.206.126:42133 requested a full tablet report, sending...
I20260925 13:07:10.756332 1395 ts_manager.cc:194] Registered new tserver with Master: 56833387211f414aa1790ab2abff49f7 (127.0.206.66:45291)
I20260925 13:07:10.756837 1395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:51817
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260925 13:07:10.772990 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:42133
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:10.850324 1725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:10.850502 1725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:10.850525 1725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:10.851708 1725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:10.851747 1725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:10.851773 1725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:07:10.853092 1725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:42133
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:10.853277 1725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:10.853539 1725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:10.858201 1731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:10.858237 1725 server_base.cc:1034] running on GCE node
W20260925 13:07:10.858237 1733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:10.858199 1730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:10.858611 1725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:10.858924 1725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:10.860080 1725 hybrid_clock.cc:648] HybridClock initialized: now 1790341630860062 us; error 47 us; skew 500 ppm
Sep 25 13:07:10 dist-test-slave-chkf krb5kdc[1350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341630, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:10.862388 1725 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:07:10.862954 1725 webserver.cc:460] Webserver started at http://127.0.206.67:44891/ using document root <none> and password file <none>
I20260925 13:07:10.863179 1725 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:10.863243 1725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:10.863374 1725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:10.864355 1725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "3e90464932f74ced85006e9a18b2cedc"
format_stamp: "Formatted at 2026-09-25 13:07:10 on dist-test-slave-chkf"
server_key: "de433c08d37b17273c2855708b40ee39"
server_key_iv: "e974a6dc467004248928ebe8bde3607f"
server_key_version: "encryptionkey@0"
I20260925 13:07:10.864708 1725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "3e90464932f74ced85006e9a18b2cedc"
format_stamp: "Formatted at 2026-09-25 13:07:10 on dist-test-slave-chkf"
server_key: "de433c08d37b17273c2855708b40ee39"
server_key_iv: "e974a6dc467004248928ebe8bde3607f"
server_key_version: "encryptionkey@0"
I20260925 13:07:10.866000 1725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:07:10.866729 1740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:10.866904 1725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:10.867009 1725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "3e90464932f74ced85006e9a18b2cedc"
format_stamp: "Formatted at 2026-09-25 13:07:10 on dist-test-slave-chkf"
server_key: "de433c08d37b17273c2855708b40ee39"
server_key_iv: "e974a6dc467004248928ebe8bde3607f"
server_key_version: "encryptionkey@0"
I20260925 13:07:10.867093 1725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:10.883394 1725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:10.885069 1725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:10.885313 1725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:10.885519 1725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:10.885787 1725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:10.885836 1725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:10.885874 1725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:10.885902 1725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:10.891386 1725 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:39551
I20260925 13:07:10.891428 1853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:39551 every 8 connection(s)
I20260925 13:07:10.891712 1725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:07:10.896970 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 1725
I20260925 13:07:10.897049 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:07:10.897164 825 external_mini_cluster.cc:1428] Setting key f4691622f9513d0d16027f5aa16ac413
Sep 25 13:07:10 dist-test-slave-chkf krb5kdc[1350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341630, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:10.900666 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:10.892375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:38325 (local address 127.0.206.126:42133)
0925 13:07:10.892434 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:10.892437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:10.892602 (+ 165us) server_negotiation.cc:407] Connection header received
0925 13:07:10.893079 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:10.893080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:10.893091 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:10.893155 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:10.894102 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.894595 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.895639 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.895832 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.895935 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:10.897963 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:10.897965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:10.897966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:10.897973 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:10.899369 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:10.899762 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:10.899763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:10.899763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:10.899820 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:10.900069 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:10.900070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:10.900070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:10.900175 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:10.900242 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:10.900502 (+ 260us) server_negotiation.cc:299] Negotiation successful
0925 13:07:10.900579 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:07:10.900835 1856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:10.892340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:42133 (local address 127.0.206.67:38325)
0925 13:07:10.892484 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:10.892524 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:10.892978 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:10.893216 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:10.893220 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:10.893817 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:10.893993 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:10.893997 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.894685 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.894686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:10.895539 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:10.895541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.895939 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.895941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:10.896071 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:10.896534 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:10.896541 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:10.897876 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:10.899455 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:10.899462 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:10.899464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:10.899674 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:10.899885 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:10.899886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:10.899886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:10.899995 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:10.900245 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:10.900252 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:10.900377 (+ 125us) client_negotiation.cc:769] Sending connection context
0925 13:07:10.900479 (+ 102us) client_negotiation.cc:240] Negotiation successful
0925 13:07:10.900620 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260925 13:07:10.901266 1854 heartbeater.cc:347] Connected to a master server at 127.0.206.126:42133
I20260925 13:07:10.901332 1854 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:10.901508 1854 heartbeater.cc:511] Master 127.0.206.126:42133 requested a full tablet report, sending...
I20260925 13:07:10.901827 1395 ts_manager.cc:194] Registered new tserver with Master: 3e90464932f74ced85006e9a18b2cedc (127.0.206.67:39551)
I20260925 13:07:10.902179 1395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:38325
I20260925 13:07:10.908772 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:07:10.915294 1451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:10.910180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48388 (local address 127.0.206.126:42133)
0925 13:07:10.910233 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:10.910235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:10.910300 (+ 65us) server_negotiation.cc:407] Connection header received
0925 13:07:10.910333 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:10.910333 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:10.910341 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:10.910382 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:10.910916 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.911340 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.911980 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.912143 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.912188 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:10.913137 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:10.913138 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:10.913139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:10.913144 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:10.914433 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:10.914748 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:10.914749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:10.914749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:10.914793 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:10.914925 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:10.914926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:10.914926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:10.915032 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:10.915107 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:10.915193 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 13:07:10.915242 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 13:07:10.917078 1395 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:07:10.918480 1395 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:07:10.924328 1862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:10.920363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:36509 (local address 127.0.0.1:54084)
0925 13:07:10.920476 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:10.920510 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:10.920584 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:10.920903 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:10.920905 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:10.920906 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:10.920969 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:10.920970 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.922250 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.922250 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:10.923319 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:10.923320 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.924025 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.924026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:10.924110 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:10.924122 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 13:07:10.924212 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 13:07:10.924263 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":72,"threads_started":1}
I20260925 13:07:10.924336 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:10.920560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54084 (local address 127.0.206.65:36509)
0925 13:07:10.920760 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:10.920764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:10.920773 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:10.920811 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:10.920813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:10.920828 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:10.920901 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:10.921055 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.922157 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.923376 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.923938 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.924027 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:10.924080 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:10.924227 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 13:07:10.924263 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":65,"threads_started":1}
I20260925 13:07:10.924443 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:10.920497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52168 (local address 127.0.206.66:45291)
0925 13:07:10.920685 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:10.920690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:10.920791 (+ 101us) server_negotiation.cc:407] Connection header received
0925 13:07:10.920863 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:10.920866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:10.920884 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:10.920966 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:10.921098 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.921988 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.923386 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.923999 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.924045 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:10.924092 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:10.924298 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 13:07:10.924343 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":71,"threads_started":1}
I20260925 13:07:10.924661 1866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:10.920606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:39551 (local address 127.0.0.1:45286)
0925 13:07:10.920991 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:10.921001 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:10.921051 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:10.921302 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:10.921303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:10.921305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:10.921347 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:10.921348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.922571 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.922572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:10.923552 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:10.923554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.924442 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.924443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:10.924508 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:10.924512 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:07:10.924563 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 13:07:10.924592 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":47,"threads_started":1}
I20260925 13:07:10.924660 1863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:10.920388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:45291 (local address 127.0.0.1:52168)
0925 13:07:10.920716 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:10.920727 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:10.920792 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:10.920963 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:10.920964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:10.920966 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:10.921012 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:10.921014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.922085 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.922086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:10.923286 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:10.923288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.924150 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.924151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:10.924210 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:10.924213 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:07:10.924542 (+ 329us) client_negotiation.cc:240] Negotiation successful
0925 13:07:10.924586 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":48,"threads_started":1}
I20260925 13:07:10.924875 1867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:10.920764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45286 (local address 127.0.206.67:39551)
0925 13:07:10.920902 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:10.920907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:10.921046 (+ 139us) server_negotiation.cc:407] Connection header received
0925 13:07:10.921108 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:10.921110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:10.921125 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:10.921197 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:10.921478 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.922428 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.923630 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:10.924280 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:10.924336 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:10.924380 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:10.924761 (+ 381us) server_negotiation.cc:299] Negotiation successful
0925 13:07:10.924808 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260925 13:07:10.925284 1520 tablet_service.cc:1467] Processing CreateTablet for tablet 40a832e2c95f4c928bc9a1fcbe9c2ba3 (DEFAULT_TABLE table=test-table [id=28881ad06448451582e2feb128fd7783]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:10.925540 1788 tablet_service.cc:1467] Processing CreateTablet for tablet 40a832e2c95f4c928bc9a1fcbe9c2ba3 (DEFAULT_TABLE table=test-table [id=28881ad06448451582e2feb128fd7783]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:10.925644 1520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40a832e2c95f4c928bc9a1fcbe9c2ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:10.925853 1788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40a832e2c95f4c928bc9a1fcbe9c2ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:10.928085 1868 tablet_bootstrap.cc:492] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930: Bootstrap starting.
I20260925 13:07:10.928253 1869 tablet_bootstrap.cc:492] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 3e90464932f74ced85006e9a18b2cedc: Bootstrap starting.
I20260925 13:07:10.928545 1654 tablet_service.cc:1467] Processing CreateTablet for tablet 40a832e2c95f4c928bc9a1fcbe9c2ba3 (DEFAULT_TABLE table=test-table [id=28881ad06448451582e2feb128fd7783]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:10.928789 1868 tablet_bootstrap.cc:654] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:10.928855 1654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40a832e2c95f4c928bc9a1fcbe9c2ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:10.928900 1869 tablet_bootstrap.cc:654] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 3e90464932f74ced85006e9a18b2cedc: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:10.929368 1869 log.cc:824] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 3e90464932f74ced85006e9a18b2cedc: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:10.929364 1868 log.cc:824] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:10.930039 1869 tablet_bootstrap.cc:492] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 3e90464932f74ced85006e9a18b2cedc: No bootstrap required, opened a new log
I20260925 13:07:10.930092 1868 tablet_bootstrap.cc:492] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930: No bootstrap required, opened a new log
I20260925 13:07:10.930116 1869 ts_tablet_manager.cc:1397] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 3e90464932f74ced85006e9a18b2cedc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:07:10.930156 1868 ts_tablet_manager.cc:1397] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:07:10.931542 1869 raft_consensus.cc:348] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 3e90464932f74ced85006e9a18b2cedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad1aec33a1da4fee9b4e1eac53ae6930" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36509 } } peers { permanent_uuid: "56833387211f414aa1790ab2abff49f7" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45291 } } peers { permanent_uuid: "3e90464932f74ced85006e9a18b2cedc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39551 } }
I20260925 13:07:10.931689 1869 raft_consensus.cc:374] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 3e90464932f74ced85006e9a18b2cedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:10.931728 1869 raft_consensus.cc:729] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 3e90464932f74ced85006e9a18b2cedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e90464932f74ced85006e9a18b2cedc, State: Initialized, Role: FOLLOWER
I20260925 13:07:10.931852 1869 consensus_queue.cc:260] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 3e90464932f74ced85006e9a18b2cedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad1aec33a1da4fee9b4e1eac53ae6930" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36509 } } peers { permanent_uuid: "56833387211f414aa1790ab2abff49f7" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45291 } } peers { permanent_uuid: "3e90464932f74ced85006e9a18b2cedc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39551 } }
I20260925 13:07:10.932097 1869 ts_tablet_manager.cc:1428] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 3e90464932f74ced85006e9a18b2cedc: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 13:07:10.932224 1854 heartbeater.cc:503] Master 127.0.206.126:42133 was elected leader, sending a full tablet report...
I20260925 13:07:10.932555 1868 raft_consensus.cc:348] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad1aec33a1da4fee9b4e1eac53ae6930" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36509 } } peers { permanent_uuid: "56833387211f414aa1790ab2abff49f7" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45291 } } peers { permanent_uuid: "3e90464932f74ced85006e9a18b2cedc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39551 } }
I20260925 13:07:10.932710 1868 raft_consensus.cc:374] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:10.932752 1868 raft_consensus.cc:729] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad1aec33a1da4fee9b4e1eac53ae6930, State: Initialized, Role: FOLLOWER
I20260925 13:07:10.932854 1868 consensus_queue.cc:260] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad1aec33a1da4fee9b4e1eac53ae6930" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36509 } } peers { permanent_uuid: "56833387211f414aa1790ab2abff49f7" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45291 } } peers { permanent_uuid: "3e90464932f74ced85006e9a18b2cedc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39551 } }
I20260925 13:07:10.932876 1873 tablet_bootstrap.cc:492] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7: Bootstrap starting.
I20260925 13:07:10.933123 1868 ts_tablet_manager.cc:1428] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 13:07:10.933143 1586 heartbeater.cc:503] Master 127.0.206.126:42133 was elected leader, sending a full tablet report...
I20260925 13:07:10.933665 1873 tablet_bootstrap.cc:654] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:10.934192 1873 log.cc:824] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:10.934787 1873 tablet_bootstrap.cc:492] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7: No bootstrap required, opened a new log
I20260925 13:07:10.934849 1873 ts_tablet_manager.cc:1397] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:07:10.936062 1873 raft_consensus.cc:348] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad1aec33a1da4fee9b4e1eac53ae6930" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36509 } } peers { permanent_uuid: "56833387211f414aa1790ab2abff49f7" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45291 } } peers { permanent_uuid: "3e90464932f74ced85006e9a18b2cedc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39551 } }
I20260925 13:07:10.936182 1873 raft_consensus.cc:374] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:10.936203 1873 raft_consensus.cc:729] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56833387211f414aa1790ab2abff49f7, State: Initialized, Role: FOLLOWER
I20260925 13:07:10.936303 1873 consensus_queue.cc:260] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad1aec33a1da4fee9b4e1eac53ae6930" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36509 } } peers { permanent_uuid: "56833387211f414aa1790ab2abff49f7" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45291 } } peers { permanent_uuid: "3e90464932f74ced85006e9a18b2cedc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39551 } }
I20260925 13:07:10.936542 1873 ts_tablet_manager.cc:1428] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:07:10.936569 1720 heartbeater.cc:503] Master 127.0.206.126:42133 was elected leader, sending a full tablet report...
W20260925 13:07:10.996482 1721 tablet.cc:2369] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:07:11.009315 1874 raft_consensus.cc:482] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:11.009451 1874 raft_consensus.cc:504] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad1aec33a1da4fee9b4e1eac53ae6930" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36509 } } peers { permanent_uuid: "56833387211f414aa1790ab2abff49f7" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45291 } } peers { permanent_uuid: "3e90464932f74ced85006e9a18b2cedc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39551 } }
I20260925 13:07:11.009851 1874 leader_election.cc:288] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56833387211f414aa1790ab2abff49f7 (127.0.206.66:45291), 3e90464932f74ced85006e9a18b2cedc (127.0.206.67:39551)
I20260925 13:07:11.011623 1876 raft_consensus.cc:482] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:11.011719 1876 raft_consensus.cc:504] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad1aec33a1da4fee9b4e1eac53ae6930" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36509 } } peers { permanent_uuid: "56833387211f414aa1790ab2abff49f7" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45291 } } peers { permanent_uuid: "3e90464932f74ced85006e9a18b2cedc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39551 } }
I20260925 13:07:11.012002 1876 leader_election.cc:288] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad1aec33a1da4fee9b4e1eac53ae6930 (127.0.206.65:36509), 3e90464932f74ced85006e9a18b2cedc (127.0.206.67:39551)
I20260925 13:07:11.013245 1588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:11.010044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:45291 (local address 127.0.206.65:49937)
0925 13:07:11.010094 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:11.010107 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:11.010153 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:11.010333 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:11.010335 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:11.010337 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:11.010411 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:11.010413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.011186 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.011187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:11.012179 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:11.012182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.012998 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.012999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:11.013079 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:11.013083 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:07:11.013129 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 13:07:11.013173 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 13:07:11.013266 1867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:11.010204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:53795 (local address 127.0.206.67:39551)
0925 13:07:11.010255 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:11.010257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:11.010328 (+ 71us) server_negotiation.cc:407] Connection header received
0925 13:07:11.010374 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:11.010374 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:11.010385 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:11.010446 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:11.010601 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.011080 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.012333 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.012916 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.012963 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:11.012971 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:11.013168 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 13:07:11.013204 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 13:07:11.013484 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:11.010106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:49937 (local address 127.0.206.66:45291)
0925 13:07:11.010154 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:11.010156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:11.010165 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:11.010232 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:11.010233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:11.010245 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:11.010302 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:11.010472 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.011086 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.012280 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.012882 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.012947 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:11.012969 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:11.013393 (+ 424us) server_negotiation.cc:299] Negotiation successful
0925 13:07:11.013430 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 13:07:11.013523 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:11.010091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:39551 (local address 127.0.206.65:53795)
0925 13:07:11.010259 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:11.010268 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:11.010320 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:11.010465 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:11.010466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:11.010467 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:11.010533 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:11.010535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.011153 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.011154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:11.012213 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:11.012215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.013025 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.013026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:11.013096 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:11.013099 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:07:11.013362 (+ 263us) client_negotiation.cc:240] Negotiation successful
0925 13:07:11.013409 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":31,"thread_start_us":96,"threads_started":1}
I20260925 13:07:11.013717 1674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a832e2c95f4c928bc9a1fcbe9c2ba3" candidate_uuid: "ad1aec33a1da4fee9b4e1eac53ae6930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56833387211f414aa1790ab2abff49f7" is_pre_election: true
I20260925 13:07:11.013795 1674 raft_consensus.cc:2454] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad1aec33a1da4fee9b4e1eac53ae6930 in term 0.
I20260925 13:07:11.013792 1808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a832e2c95f4c928bc9a1fcbe9c2ba3" candidate_uuid: "ad1aec33a1da4fee9b4e1eac53ae6930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e90464932f74ced85006e9a18b2cedc" is_pre_election: true
I20260925 13:07:11.013923 1808 raft_consensus.cc:2454] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 3e90464932f74ced85006e9a18b2cedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad1aec33a1da4fee9b4e1eac53ae6930 in term 0.
I20260925 13:07:11.013978 1475 leader_election.cc:302] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56833387211f414aa1790ab2abff49f7, ad1aec33a1da4fee9b4e1eac53ae6930; no voters:
I20260925 13:07:11.014122 1874 raft_consensus.cc:2781] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:07:11.014158 1874 raft_consensus.cc:482] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:07:11.014183 1874 raft_consensus.cc:3037] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:11.015055 1874 raft_consensus.cc:504] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad1aec33a1da4fee9b4e1eac53ae6930" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36509 } } peers { permanent_uuid: "56833387211f414aa1790ab2abff49f7" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45291 } } peers { permanent_uuid: "3e90464932f74ced85006e9a18b2cedc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39551 } }
I20260925 13:07:11.015300 1674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a832e2c95f4c928bc9a1fcbe9c2ba3" candidate_uuid: "ad1aec33a1da4fee9b4e1eac53ae6930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56833387211f414aa1790ab2abff49f7"
I20260925 13:07:11.015372 1674 raft_consensus.cc:3037] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:11.015506 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:11.012180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:36509 (local address 127.0.206.66:44025)
0925 13:07:11.012323 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:11.012333 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:11.012374 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:11.012554 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:11.012555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:11.012557 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:11.012615 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:11.012616 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.013843 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.013843 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:11.014641 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:11.014643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.015273 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.015273 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:11.015342 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:11.015344 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:07:11.015409 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 13:07:11.015452 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260925 13:07:11.015744 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:11.012240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:44025 (local address 127.0.206.65:36509)
0925 13:07:11.012276 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:11.012277 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:11.012468 (+ 191us) server_negotiation.cc:407] Connection header received
0925 13:07:11.012488 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:11.012488 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:11.012497 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:11.012545 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:11.012693 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.013336 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.014731 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.015194 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.015273 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:11.015281 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:11.015663 (+ 382us) server_negotiation.cc:299] Negotiation successful
0925 13:07:11.015699 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 13:07:11.015992 1540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a832e2c95f4c928bc9a1fcbe9c2ba3" candidate_uuid: "56833387211f414aa1790ab2abff49f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad1aec33a1da4fee9b4e1eac53ae6930" is_pre_election: true
I20260925 13:07:11.016088 1540 raft_consensus.cc:2379] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56833387211f414aa1790ab2abff49f7 in current term 1: Already voted for candidate ad1aec33a1da4fee9b4e1eac53ae6930 in this term.
I20260925 13:07:11.016503 1878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:11.012310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:39551 (local address 127.0.206.66:47245)
0925 13:07:11.012601 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:11.012610 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:11.012645 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:11.012984 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:11.012985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:11.012987 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:11.013045 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:11.013046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.013924 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.013925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:11.015060 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:11.015063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.015887 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.015888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:11.015953 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:11.015956 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:07:11.015986 (+ 30us) client_negotiation.cc:240] Negotiation successful
0925 13:07:11.016019 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":2,"thread_start_us":81,"threads_started":1}
I20260925 13:07:11.016780 1879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:11.012722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:47245 (local address 127.0.206.67:39551)
0925 13:07:11.012861 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:11.012863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:11.012869 (+ 6us) server_negotiation.cc:407] Connection header received
0925 13:07:11.012888 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:11.012889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:11.012898 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:11.013106 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:11.013138 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.013831 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.015160 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.015792 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.016617 (+ 825us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:11.016626 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:11.016649 (+ 23us) server_negotiation.cc:299] Negotiation successful
0925 13:07:11.016692 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260925 13:07:11.016793 1674 raft_consensus.cc:2454] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad1aec33a1da4fee9b4e1eac53ae6930 in term 1.
I20260925 13:07:11.016893 1874 leader_election.cc:288] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 [CANDIDATE]: Term 1 election: Requested vote from peers 56833387211f414aa1790ab2abff49f7 (127.0.206.66:45291), 3e90464932f74ced85006e9a18b2cedc (127.0.206.67:39551)
I20260925 13:07:11.016935 1475 leader_election.cc:302] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56833387211f414aa1790ab2abff49f7, ad1aec33a1da4fee9b4e1eac53ae6930; no voters:
I20260925 13:07:11.017017 1874 raft_consensus.cc:2781] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:11.017037 1808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a832e2c95f4c928bc9a1fcbe9c2ba3" candidate_uuid: "56833387211f414aa1790ab2abff49f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e90464932f74ced85006e9a18b2cedc" is_pre_election: true
I20260925 13:07:11.017129 1874 raft_consensus.cc:686] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 [term 1 LEADER]: Becoming Leader. State: Replica: ad1aec33a1da4fee9b4e1eac53ae6930, State: Running, Role: LEADER
I20260925 13:07:11.017168 1808 raft_consensus.cc:2454] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 3e90464932f74ced85006e9a18b2cedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56833387211f414aa1790ab2abff49f7 in term 0.
I20260925 13:07:11.017050 1807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a832e2c95f4c928bc9a1fcbe9c2ba3" candidate_uuid: "ad1aec33a1da4fee9b4e1eac53ae6930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e90464932f74ced85006e9a18b2cedc"
I20260925 13:07:11.017205 1874 consensus_queue.cc:237] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad1aec33a1da4fee9b4e1eac53ae6930" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36509 } } peers { permanent_uuid: "56833387211f414aa1790ab2abff49f7" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45291 } } peers { permanent_uuid: "3e90464932f74ced85006e9a18b2cedc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39551 } }
I20260925 13:07:11.017248 1807 raft_consensus.cc:3037] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 3e90464932f74ced85006e9a18b2cedc [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:11.017715 1608 leader_election.cc:302] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e90464932f74ced85006e9a18b2cedc, 56833387211f414aa1790ab2abff49f7; no voters: ad1aec33a1da4fee9b4e1eac53ae6930
I20260925 13:07:11.017838 1876 raft_consensus.cc:2741] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 56833387211f414aa1790ab2abff49f7 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260925 13:07:11.018181 1807 raft_consensus.cc:2454] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P 3e90464932f74ced85006e9a18b2cedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad1aec33a1da4fee9b4e1eac53ae6930 in term 1.
I20260925 13:07:11.018369 1395 catalog_manager.cc:5495] T 40a832e2c95f4c928bc9a1fcbe9c2ba3 P ad1aec33a1da4fee9b4e1eac53ae6930 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad1aec33a1da4fee9b4e1eac53ae6930 (127.0.206.65). New cstate: current_term: 1 leader_uuid: "ad1aec33a1da4fee9b4e1eac53ae6930" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad1aec33a1da4fee9b4e1eac53ae6930" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56833387211f414aa1790ab2abff49f7" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e90464932f74ced85006e9a18b2cedc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39551 } health_report { overall_health: UNKNOWN } } }
Sep 25 13:07:11 dist-test-slave-chkf krb5kdc[1350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260925 13:07:11.047140 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:11.040191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54096 (local address 127.0.206.65:36509)
0925 13:07:11.040253 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:11.040255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:11.040362 (+ 107us) server_negotiation.cc:407] Connection header received
0925 13:07:11.040447 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:11.040448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:11.040456 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:11.040501 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:11.041033 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.041465 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.042143 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.042318 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.042395 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:11.044559 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:11.044566 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:11.044566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:11.044572 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:11.046163 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:11.046450 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:11.046452 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:11.046453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:11.046512 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:11.046651 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:11.046652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:11.046652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:11.046822 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:11.046929 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:11.046982 (+ 53us) server_negotiation.cc:299] Negotiation successful
0925 13:07:11.047038 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Sep 25 13:07:11 dist-test-slave-chkf krb5kdc[1350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341631, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 13:07:11 dist-test-slave-chkf krb5kdc[1350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341631, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260925 13:07:11.068184 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:11.061763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54100 (local address 127.0.206.65:36509)
0925 13:07:11.061821 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:11.061823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:11.061918 (+ 95us) server_negotiation.cc:407] Connection header received
0925 13:07:11.061974 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:11.061974 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:11.061982 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:11.062027 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:11.062508 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.063004 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.063646 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.063825 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.063871 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:11.065816 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:11.065817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:11.065817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:11.065828 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:11.067160 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:11.067424 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:11.067425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:11.067425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:11.067478 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:11.067644 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:11.067645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:11.067646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:11.067744 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:11.067862 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:11.068015 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 13:07:11.068087 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Sep 25 13:07:11 dist-test-slave-chkf krb5kdc[1350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 13:07:11 dist-test-slave-chkf krb5kdc[1350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260925 13:07:11.089603 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:11.082788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54114 (local address 127.0.206.65:36509)
0925 13:07:11.082865 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:11.082885 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:11.083007 (+ 122us) server_negotiation.cc:407] Connection header received
0925 13:07:11.083084 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:11.083085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:11.083094 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:11.083178 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:11.083677 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.084128 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.084858 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:11.085040 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:11.085123 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:11.087349 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:11.087350 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:11.087351 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:11.087357 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:11.088478 (+ 1121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:11.088847 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:11.088848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:11.088849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:11.088901 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:11.089101 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:11.089102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:11.089102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:11.089194 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:11.089295 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:11.089425 (+ 130us) server_negotiation.cc:299] Negotiation successful
0925 13:07:11.089491 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260925 13:07:11.090016 1500 tablet_service.cc:2908] Rejecting scan request for tablet 40a832e2c95f4c928bc9a1fcbe9c2ba3: Uninitialized: clean time has not yet been initialized
I20260925 13:07:11.090775 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 1457
I20260925 13:07:11.095646 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 1591
I20260925 13:07:11.100049 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 1725
I20260925 13:07:11.104305 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 1363
2026-09-25T13:07:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1966 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:07:11 dist-test-slave-chkf krb5kdc[1894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:07:11 dist-test-slave-chkf krb5kdc[1894](info): set up 2 sockets
Sep 25 13:07:11 dist-test-slave-chkf krb5kdc[1894](info): commencing operation
krb5kdc: starting...
W20260925 13:07:12.122201 825 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:07:12 dist-test-slave-chkf krb5kdc[1894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:07:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:07:12.233058 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:33717
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:44523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:33717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:12.309478 1907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:12.309653 1907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:12.309675 1907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:12.310799 1907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:12.310837 1907 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 13:07:12.310849 1907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:12.310863 1907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:12.310913 1907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:12.310931 1907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:07:12.312376 1907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:44523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:33717
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:33717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:12.312597 1907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:12.312814 1907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:12.317452 1915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:12.317461 1913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:12.317480 1907 server_base.cc:1034] running on GCE node
W20260925 13:07:12.317447 1912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:12.317997 1907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:12.318280 1907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:12.319458 1907 hybrid_clock.cc:648] HybridClock initialized: now 1790341632319434 us; error 34 us; skew 500 ppm
Sep 25 13:07:12 dist-test-slave-chkf krb5kdc[1894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341632, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:12.328270 1907 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:07:12.328688 1907 webserver.cc:460] Webserver started at http://127.0.206.126:43831/ using document root <none> and password file <none>
I20260925 13:07:12.328884 1907 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:12.328925 1907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:12.329008 1907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:12.329954 1907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "f434f38b0b0a48749433f3a4e1e2f569"
format_stamp: "Formatted at 2026-09-25 13:07:12 on dist-test-slave-chkf"
server_key: "51710e9fbf8b23b31432db154355cb83"
server_key_iv: "8baa1ed75c0253a1d5f8559f27f3d690"
server_key_version: "encryptionkey@0"
I20260925 13:07:12.330243 1907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "f434f38b0b0a48749433f3a4e1e2f569"
format_stamp: "Formatted at 2026-09-25 13:07:12 on dist-test-slave-chkf"
server_key: "51710e9fbf8b23b31432db154355cb83"
server_key_iv: "8baa1ed75c0253a1d5f8559f27f3d690"
server_key_version: "encryptionkey@0"
I20260925 13:07:12.331564 1907 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 13:07:12.332242 1922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:12.332438 1907 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:12.332517 1907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "f434f38b0b0a48749433f3a4e1e2f569"
format_stamp: "Formatted at 2026-09-25 13:07:12 on dist-test-slave-chkf"
server_key: "51710e9fbf8b23b31432db154355cb83"
server_key_iv: "8baa1ed75c0253a1d5f8559f27f3d690"
server_key_version: "encryptionkey@0"
I20260925 13:07:12.332567 1907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:12.360150 1907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:12.362268 1907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:12.362375 1907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:12.366425 1907 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:33717
I20260925 13:07:12.366485 1984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:33717 every 8 connection(s)
I20260925 13:07:12.366808 1907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:12.367278 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 1907
I20260925 13:07:12.367343 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:07:12.367504 825 external_mini_cluster.cc:1428] Setting key 7b5b24b595a109993e18f13f697fe1a9
I20260925 13:07:12.367506 1985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:12.370872 1985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569: Bootstrap starting.
I20260925 13:07:12.371594 1985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:12.371922 1985 log.cc:824] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:12.372677 1985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569: No bootstrap required, opened a new log
Sep 25 13:07:12 dist-test-slave-chkf krb5kdc[1894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:12.375547 1985 raft_consensus.cc:348] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f434f38b0b0a48749433f3a4e1e2f569" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33717 } }
I20260925 13:07:12.375703 1985 raft_consensus.cc:374] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:12.375777 1985 raft_consensus.cc:729] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f434f38b0b0a48749433f3a4e1e2f569, State: Initialized, Role: FOLLOWER
I20260925 13:07:12.375910 1985 consensus_queue.cc:260] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f434f38b0b0a48749433f3a4e1e2f569" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33717 } }
I20260925 13:07:12.376008 1985 raft_consensus.cc:388] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:12.376085 1985 raft_consensus.cc:482] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:12.376137 1985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:12.376895 1985 raft_consensus.cc:504] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f434f38b0b0a48749433f3a4e1e2f569" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33717 } }
I20260925 13:07:12.377005 1985 leader_election.cc:302] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f434f38b0b0a48749433f3a4e1e2f569; no voters:
I20260925 13:07:12.377210 1985 leader_election.cc:288] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:07:12.377242 1990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:12.377458 1990 raft_consensus.cc:686] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569 [term 1 LEADER]: Becoming Leader. State: Replica: f434f38b0b0a48749433f3a4e1e2f569, State: Running, Role: LEADER
I20260925 13:07:12.377537 1985 sys_catalog.cc:564] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:12.377584 1990 consensus_queue.cc:237] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f434f38b0b0a48749433f3a4e1e2f569" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33717 } }
I20260925 13:07:12.377945 1990 sys_catalog.cc:455] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f434f38b0b0a48749433f3a4e1e2f569" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f434f38b0b0a48749433f3a4e1e2f569" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33717 } } }
I20260925 13:07:12.378118 1990 sys_catalog.cc:458] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:12.378284 1990 sys_catalog.cc:455] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f434f38b0b0a48749433f3a4e1e2f569. Latest consensus state: current_term: 1 leader_uuid: "f434f38b0b0a48749433f3a4e1e2f569" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f434f38b0b0a48749433f3a4e1e2f569" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33717 } } }
I20260925 13:07:12.378396 1990 sys_catalog.cc:458] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:12.378566 1999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:12.379130 1999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:07:12.380276 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.367991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34964 (local address 127.0.206.126:33717)
0925 13:07:12.368169 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:12.368179 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:12.368210 (+ 31us) server_negotiation.cc:407] Connection header received
0925 13:07:12.368723 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:12.368732 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:12.368791 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:12.368951 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:12.369476 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.370879 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.372040 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.372699 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.373113 (+ 414us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.375985 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:12.375989 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:12.375989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:12.376001 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:12.378191 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:12.378611 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:12.378615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:12.378615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:12.378756 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:12.379002 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:12.379003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:12.379004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:12.379288 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:12.379424 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:12.379954 (+ 530us) server_negotiation.cc:299] Negotiation successful
0925 13:07:12.380083 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260925 13:07:12.381655 1999 catalog_manager.cc:1347] Generated new cluster ID: 08f659e3f9a741fba56175fcc0828fa1
I20260925 13:07:12.381714 1999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:07:12.391057 1999 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:07:12.391743 1999 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:12.398891 1999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569: Generated new TSK 0
I20260925 13:07:12.399044 1999 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260925 13:07:12.425557 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33717
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:44523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:12.505017 2011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:12.505191 2011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:12.505213 2011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:12.506274 2011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:12.506305 2011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:12.506325 2011 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 13:07:12.506341 2011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:07:12.507742 2011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:44523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.206.126:33717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:12.507920 2011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:12.508077 2011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:12.512656 2019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:12.512646 2016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:12.512667 2017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:12.512696 2011 server_base.cc:1034] running on GCE node
I20260925 13:07:12.513092 2011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:12.513319 2011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:12.514468 2011 hybrid_clock.cc:648] HybridClock initialized: now 1790341632514457 us; error 28 us; skew 500 ppm
Sep 25 13:07:12 dist-test-slave-chkf krb5kdc[1894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341632, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:12.516747 2011 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:07:12.517202 2011 webserver.cc:460] Webserver started at http://127.0.206.65:36033/ using document root <none> and password file <none>
I20260925 13:07:12.517414 2011 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:12.517488 2011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:12.517635 2011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:12.518608 2011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "5a3f81f5c0ab48fb92be9039866d7768"
format_stamp: "Formatted at 2026-09-25 13:07:12 on dist-test-slave-chkf"
server_key: "386b5df861262713e036704cc77acf9c"
server_key_iv: "a98ca8f36c20939310c9d828fb015b6b"
server_key_version: "encryptionkey@0"
I20260925 13:07:12.518996 2011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "5a3f81f5c0ab48fb92be9039866d7768"
format_stamp: "Formatted at 2026-09-25 13:07:12 on dist-test-slave-chkf"
server_key: "386b5df861262713e036704cc77acf9c"
server_key_iv: "a98ca8f36c20939310c9d828fb015b6b"
server_key_version: "encryptionkey@0"
I20260925 13:07:12.520231 2011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:12.520891 2026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:12.521060 2011 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:12.521162 2011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "5a3f81f5c0ab48fb92be9039866d7768"
format_stamp: "Formatted at 2026-09-25 13:07:12 on dist-test-slave-chkf"
server_key: "386b5df861262713e036704cc77acf9c"
server_key_iv: "a98ca8f36c20939310c9d828fb015b6b"
server_key_version: "encryptionkey@0"
I20260925 13:07:12.521241 2011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:12.536593 2011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:12.538331 2011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:12.538479 2011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:12.539034 2011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:12.539091 2011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:12.539143 2011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:12.539170 2011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:12.545657 2011 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:37717
I20260925 13:07:12.545740 2141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:37717 every 8 connection(s)
I20260925 13:07:12.546017 2011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
Sep 25 13:07:12 dist-test-slave-chkf krb5kdc[1894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341632, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:12.548756 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.539517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:52927 (local address 127.0.206.126:33717)
0925 13:07:12.539594 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:12.539596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:12.539884 (+ 288us) server_negotiation.cc:407] Connection header received
0925 13:07:12.540376 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:12.540377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:12.540389 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:12.540434 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:12.541274 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.542113 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.543750 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.544086 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.544179 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.546588 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:12.546589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:12.546589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:12.546595 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:12.547734 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:12.548081 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:12.548082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:12.548083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:12.548131 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:12.548308 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:12.548309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:12.548309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:12.548398 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:12.548445 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:12.548640 (+ 195us) server_negotiation.cc:299] Negotiation successful
0925 13:07:12.548696 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 13:07:12.548985 2035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.539618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33717 (local address 127.0.206.65:52927)
0925 13:07:12.539772 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:12.539803 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:12.540246 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:12.540498 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:12.540517 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:12.540892 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:12.541169 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:12.541174 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.542219 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.542220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:12.543625 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:12.543628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.544178 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.544179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:12.544270 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.544826 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:12.544835 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:12.546510 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:12.547812 (+ 1302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:12.547820 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:12.547822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:12.548017 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:12.548171 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:12.548172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:12.548172 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:12.548259 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:12.548453 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:12.548457 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:12.548555 (+ 98us) client_negotiation.cc:769] Sending connection context
0925 13:07:12.548630 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 13:07:12.548733 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260925 13:07:12.549467 2142 heartbeater.cc:347] Connected to a master server at 127.0.206.126:33717
I20260925 13:07:12.549566 2142 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:12.549619 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 2011
I20260925 13:07:12.549711 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:07:12.549733 2142 heartbeater.cc:511] Master 127.0.206.126:33717 requested a full tablet report, sending...
I20260925 13:07:12.549888 825 external_mini_cluster.cc:1428] Setting key 124177d24b0c0d39ca1c5a66ed50e5b6
I20260925 13:07:12.550344 1938 ts_manager.cc:194] Registered new tserver with Master: 5a3f81f5c0ab48fb92be9039866d7768 (127.0.206.65:37717)
I20260925 13:07:12.551079 1938 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:52927
I20260925 13:07:12.556466 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.550580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:42811 (local address 127.0.206.126:33717)
0925 13:07:12.550612 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:12.550614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:12.550678 (+ 64us) server_negotiation.cc:407] Connection header received
0925 13:07:12.550798 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:12.550799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:12.550808 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:12.550836 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:12.551236 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.551923 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.552763 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.552949 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.553034 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.553935 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:12.553936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:12.553936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:12.553942 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:12.555349 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:12.555805 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:12.555806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:12.555806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:12.555857 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:12.556067 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:12.556068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:12.556068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:12.556154 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:12.556203 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:12.556362 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 13:07:12.556412 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:07:12.556591 2148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.550513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33717 (local address 127.0.206.65:42811)
0925 13:07:12.550621 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:12.550631 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:12.550716 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:12.550894 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:12.550895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:12.551095 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:12.551154 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:12.551155 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.552052 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.552052 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:12.552673 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:12.552675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.553035 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.553036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:12.553101 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.553449 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:12.553466 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:12.553854 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:12.555433 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:12.555434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:12.555435 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:12.555724 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:12.555929 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:12.555929 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:12.555930 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:12.556001 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:12.556216 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:12.556217 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:12.556267 (+ 50us) client_negotiation.cc:769] Sending connection context
0925 13:07:12.556348 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 13:07:12.556442 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260925 13:07:12.573248 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33717
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:44523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:12.649343 2151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:12.649508 2151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:12.649531 2151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:12.650832 2151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:12.650870 2151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:12.650908 2151 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 13:07:12.650925 2151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:07:12.652603 2151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:44523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.206.126:33717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:12.652789 2151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:12.652947 2151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:07:12.657790 2151 server_base.cc:1034] running on GCE node
W20260925 13:07:12.657745 2159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:12.657759 2156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:12.657758 2157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:12.658097 2151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:12.658366 2151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:12.659521 2151 hybrid_clock.cc:648] HybridClock initialized: now 1790341632659511 us; error 41 us; skew 500 ppm
Sep 25 13:07:12 dist-test-slave-chkf krb5kdc[1894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341632, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:12.661881 2151 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:07:12.662273 2151 webserver.cc:460] Webserver started at http://127.0.206.66:32865/ using document root <none> and password file <none>
I20260925 13:07:12.662500 2151 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:12.662565 2151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:12.662748 2151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:12.663681 2151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "a3c3b4cc3dd14f8183a786373a29ff20"
format_stamp: "Formatted at 2026-09-25 13:07:12 on dist-test-slave-chkf"
server_key: "d9a6cc1a761d1279ecf1df4f971e37c0"
server_key_iv: "534bec4455715d0c9392a9ded0fb12fc"
server_key_version: "encryptionkey@0"
I20260925 13:07:12.664017 2151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "a3c3b4cc3dd14f8183a786373a29ff20"
format_stamp: "Formatted at 2026-09-25 13:07:12 on dist-test-slave-chkf"
server_key: "d9a6cc1a761d1279ecf1df4f971e37c0"
server_key_iv: "534bec4455715d0c9392a9ded0fb12fc"
server_key_version: "encryptionkey@0"
I20260925 13:07:12.665242 2151 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 13:07:12.665863 2166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:12.666024 2151 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:07:12.666113 2151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "a3c3b4cc3dd14f8183a786373a29ff20"
format_stamp: "Formatted at 2026-09-25 13:07:12 on dist-test-slave-chkf"
server_key: "d9a6cc1a761d1279ecf1df4f971e37c0"
server_key_iv: "534bec4455715d0c9392a9ded0fb12fc"
server_key_version: "encryptionkey@0"
I20260925 13:07:12.666188 2151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:12.697225 2151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:12.699460 2151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:12.699612 2151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:12.700111 2151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:12.700172 2151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:12.700205 2151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:12.700237 2151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 13:07:12 dist-test-slave-chkf krb5kdc[1894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341632, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:12.707429 2151 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:34789
I20260925 13:07:12.707638 2281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:34789 every 8 connection(s)
I20260925 13:07:12.707828 2151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:07:12.710108 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.700632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:56565 (local address 127.0.206.126:33717)
0925 13:07:12.700701 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:12.700703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:12.700942 (+ 239us) server_negotiation.cc:407] Connection header received
0925 13:07:12.701656 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:12.701657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:12.701667 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:12.701751 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:12.702641 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.703207 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.704818 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.705033 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.705119 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.707573 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:12.707574 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:12.707574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:12.707581 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:12.709095 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:12.709431 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:12.709432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:12.709433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:12.709482 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:12.709674 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:12.709675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:12.709675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:12.709766 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:12.709810 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:12.709994 (+ 184us) server_negotiation.cc:299] Negotiation successful
0925 13:07:12.710049 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 13:07:12.710268 2175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.700602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33717 (local address 127.0.206.66:56565)
0925 13:07:12.700800 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:12.700850 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:12.701554 (+ 704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:12.701830 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:12.701835 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:12.702248 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:12.702541 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:12.702548 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.703283 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.703283 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:12.704718 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:12.704721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.705136 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.705137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:12.705310 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.705783 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:12.705795 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:12.707475 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:12.709163 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:12.709166 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:12.709168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:12.709360 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:12.709539 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:12.709540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:12.709540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:12.709623 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:12.709808 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:12.709811 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:12.709903 (+ 92us) client_negotiation.cc:769] Sending connection context
0925 13:07:12.709982 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 13:07:12.710085 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":96,"threads_started":1}
I20260925 13:07:12.710742 2282 heartbeater.cc:347] Connected to a master server at 127.0.206.126:33717
I20260925 13:07:12.710824 2282 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:12.710999 2282 heartbeater.cc:511] Master 127.0.206.126:33717 requested a full tablet report, sending...
I20260925 13:07:12.711311 1938 ts_manager.cc:194] Registered new tserver with Master: a3c3b4cc3dd14f8183a786373a29ff20 (127.0.206.66:34789)
I20260925 13:07:12.711637 1938 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:56565
I20260925 13:07:12.716639 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.711903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:52391 (local address 127.0.206.126:33717)
0925 13:07:12.711939 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:12.711940 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:12.711948 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:12.711979 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:12.711979 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:12.711987 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:12.712055 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:12.712391 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.712832 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.713518 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.713694 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.713759 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.714543 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:12.714543 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:12.714544 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:12.714549 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:12.715859 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:12.716119 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:12.716119 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:12.716119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:12.716156 (+ 37us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:12.716288 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:12.716288 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:12.716288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:12.716393 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:12.716437 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:12.716534 (+ 97us) server_negotiation.cc:299] Negotiation successful
0925 13:07:12.716586 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:07:12.716643 2288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.711732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33717 (local address 127.0.206.66:52391)
0925 13:07:12.711865 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:12.711875 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:12.711926 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:12.712061 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:12.712061 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:12.712251 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:12.712311 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:12.712312 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.712922 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.712923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:12.713427 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:12.713429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.713770 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.713771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:12.713872 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.714202 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:12.714205 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:12.714468 (+ 263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:12.715922 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:12.715923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:12.715923 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:12.716065 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:12.716205 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:12.716205 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:12.716206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:12.716252 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:12.716441 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:12.716441 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:12.716472 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 13:07:12.716519 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 13:07:12.716582 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":78,"threads_started":1}
I20260925 13:07:12.717449 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 2151
I20260925 13:07:12.717556 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:07:12.717729 825 external_mini_cluster.cc:1428] Setting key f38ce6305c373853c6dbf565bd341dea
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260925 13:07:12.740644 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33717
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:44523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:12.817579 2291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:12.817785 2291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:12.817838 2291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:12.819020 2291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:12.819079 2291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:12.819114 2291 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 13:07:12.819169 2291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:07:12.820614 2291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:44523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.206.126:33717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:12.820859 2291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:12.821065 2291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:12.825695 2299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:12.825695 2297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:12.825862 2296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:12.825884 2291 server_base.cc:1034] running on GCE node
I20260925 13:07:12.826107 2291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:12.826272 2291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:12.827401 2291 hybrid_clock.cc:648] HybridClock initialized: now 1790341632827387 us; error 28 us; skew 500 ppm
Sep 25 13:07:12 dist-test-slave-chkf krb5kdc[1894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341632, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:12.829694 2291 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:07:12.830150 2291 webserver.cc:460] Webserver started at http://127.0.206.67:41695/ using document root <none> and password file <none>
I20260925 13:07:12.830368 2291 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:12.830411 2291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:12.830545 2291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:12.831562 2291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "bddd741031164c05a8801901c100295e"
format_stamp: "Formatted at 2026-09-25 13:07:12 on dist-test-slave-chkf"
server_key: "9056fae517406734bc0482436b5bcd9d"
server_key_iv: "c075e75988393d2fdd77df442a844bfe"
server_key_version: "encryptionkey@0"
I20260925 13:07:12.831909 2291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "bddd741031164c05a8801901c100295e"
format_stamp: "Formatted at 2026-09-25 13:07:12 on dist-test-slave-chkf"
server_key: "9056fae517406734bc0482436b5bcd9d"
server_key_iv: "c075e75988393d2fdd77df442a844bfe"
server_key_version: "encryptionkey@0"
I20260925 13:07:12.833171 2291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:07:12.833828 2306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:12.833982 2291 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:12.834084 2291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "bddd741031164c05a8801901c100295e"
format_stamp: "Formatted at 2026-09-25 13:07:12 on dist-test-slave-chkf"
server_key: "9056fae517406734bc0482436b5bcd9d"
server_key_iv: "c075e75988393d2fdd77df442a844bfe"
server_key_version: "encryptionkey@0"
I20260925 13:07:12.834151 2291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:12.847389 2291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:12.849040 2291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:12.849200 2291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:12.849745 2291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:12.849817 2291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:12.849860 2291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:12.849880 2291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 13:07:12 dist-test-slave-chkf krb5kdc[1894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341632, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:12.857061 2291 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:36335
I20260925 13:07:12.857129 2421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:36335 every 8 connection(s)
I20260925 13:07:12.857450 2291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:07:12.859210 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.850185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:36809 (local address 127.0.206.126:33717)
0925 13:07:12.850254 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:12.850257 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:12.850489 (+ 232us) server_negotiation.cc:407] Connection header received
0925 13:07:12.851191 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:12.851193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:12.851206 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:12.851260 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:12.852169 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.852667 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.853668 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.853875 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.853964 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.856525 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:12.856527 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:12.856527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:12.856534 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:12.858151 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:12.858479 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:12.858479 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:12.858480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:12.858529 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:12.858687 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:12.858687 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:12.858688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:12.858832 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:12.858882 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:12.859098 (+ 216us) server_negotiation.cc:299] Negotiation successful
0925 13:07:12.859151 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 13:07:12.859397 2315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.850169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33717 (local address 127.0.206.67:36809)
0925 13:07:12.850346 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:12.850396 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:12.851090 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:12.851358 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:12.851362 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:12.851755 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:12.852062 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:12.852069 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.852778 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.852779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:12.853576 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:12.853578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.853980 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.853981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:12.854113 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.854654 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:12.854663 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:12.856413 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:12.858217 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:12.858221 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:12.858222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:12.858414 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:12.858567 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:12.858568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:12.858568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:12.858642 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:12.858888 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:12.858892 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:12.858991 (+ 99us) client_negotiation.cc:769] Sending connection context
0925 13:07:12.859072 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 13:07:12.859185 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260925 13:07:12.859895 2422 heartbeater.cc:347] Connected to a master server at 127.0.206.126:33717
I20260925 13:07:12.859972 2422 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:12.860146 2422 heartbeater.cc:511] Master 127.0.206.126:33717 requested a full tablet report, sending...
I20260925 13:07:12.860512 1939 ts_manager.cc:194] Registered new tserver with Master: bddd741031164c05a8801901c100295e (127.0.206.67:36335)
I20260925 13:07:12.860925 1939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:36809
I20260925 13:07:12.865020 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 2291
I20260925 13:07:12.865119 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:07:12.865300 825 external_mini_cluster.cc:1428] Setting key ba7cd0cf3d6a4d1e962ea8694171e7b7
I20260925 13:07:12.866197 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.861289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:37469 (local address 127.0.206.126:33717)
0925 13:07:12.861335 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:12.861337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:12.861415 (+ 78us) server_negotiation.cc:407] Connection header received
0925 13:07:12.861479 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:12.861480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:12.861491 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:12.861547 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:12.861814 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.862257 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.862945 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.863161 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.863225 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.864006 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:12.864007 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:12.864007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:12.864012 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:12.865041 (+ 1029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:12.865444 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:12.865445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:12.865446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:12.865505 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:12.865696 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:12.865696 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:12.865697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:12.865823 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:12.865880 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:12.866035 (+ 155us) server_negotiation.cc:299] Negotiation successful
0925 13:07:12.866106 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 13:07:12.866204 2428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.861247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33717 (local address 127.0.206.67:37469)
0925 13:07:12.861360 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:12.861370 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:12.861427 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:12.861549 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:12.861549 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:12.861687 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:12.861732 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:12.861733 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.862334 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.862334 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:12.862883 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:12.862885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.863229 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.863229 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:12.863272 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.863672 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:12.863677 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:12.863928 (+ 251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:12.865137 (+ 1209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:12.865138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:12.865138 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:12.865339 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:12.865574 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:12.865575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:12.865575 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:12.865637 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:12.865887 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:12.865887 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:12.865929 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 13:07:12.865994 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 13:07:12.866093 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":70,"threads_started":1}
I20260925 13:07:12.866211 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 13:07:12 dist-test-slave-chkf krb5kdc[1894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341632, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 13:07:12 dist-test-slave-chkf krb5kdc[1894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341632, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:12.897635 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.891337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34974 (local address 127.0.206.126:33717)
0925 13:07:12.891394 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:12.891396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:12.891404 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:12.891429 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:12.891430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:12.891439 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:12.891519 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:12.892081 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.892577 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.893384 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.893593 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.893656 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.895680 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:12.895681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:12.895682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:12.895687 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:12.896784 (+ 1097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:12.897080 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:12.897081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:12.897082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:12.897137 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:12.897290 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:12.897291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:12.897291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:12.897380 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:12.897448 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:12.897524 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 13:07:12.897575 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 13:07:12.899137 1939 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:07:12.900724 1939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:07:12.907130 2443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.903133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46818 (local address 127.0.206.67:36335)
0925 13:07:12.903321 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:12.903327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:12.903338 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:07:12.903396 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:12.903398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:12.903414 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:12.903491 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:12.903637 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.904500 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.906130 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.906682 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.906811 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.906892 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:12.907006 (+ 114us) server_negotiation.cc:299] Negotiation successful
0925 13:07:12.907041 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":70,"threads_started":1}
I20260925 13:07:12.907123 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.902919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:36335 (local address 127.0.0.1:46818)
0925 13:07:12.903143 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:12.903151 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:12.903192 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:12.903502 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:12.903504 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:12.903506 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:12.903565 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:12.903566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.904573 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.904574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:12.906050 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:12.906052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.906821 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.906822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:12.906904 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.906918 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 13:07:12.906992 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 13:07:12.907041 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":5,"thread_start_us":77,"threads_started":1}
I20260925 13:07:12.907281 2440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.902925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:37717 (local address 127.0.0.1:45996)
0925 13:07:12.903322 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:12.903333 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:12.903400 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:12.904050 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:12.904051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:12.904053 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:12.904106 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:12.904107 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.905301 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.905302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:12.906031 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:12.906033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.907031 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.907032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:12.907104 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.907109 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 13:07:12.907156 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 13:07:12.907198 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":86,"threads_started":1}
I20260925 13:07:12.907382 2441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.902919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:34789 (local address 127.0.0.1:59332)
0925 13:07:12.903077 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:12.903117 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:12.903183 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:12.903762 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:12.903763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:12.903765 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:12.903825 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:12.903827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.905063 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.905064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:12.906151 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:12.906153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.907113 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.907113 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:12.907187 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.907190 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:07:12.907245 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 13:07:12.907275 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":49,"threads_started":1}
I20260925 13:07:12.907517 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.903146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59332 (local address 127.0.206.66:34789)
0925 13:07:12.903628 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:12.903632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:12.903641 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:12.903680 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:12.903682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:12.903696 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:12.903768 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:12.903905 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.904974 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.906219 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.906959 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.907021 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.907083 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:12.907285 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 13:07:12.907347 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":73,"threads_started":1}
I20260925 13:07:12.907558 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:12.903006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45996 (local address 127.0.206.65:37717)
0925 13:07:12.903899 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:12.903903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:12.903912 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:12.903950 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:12.903952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:12.903965 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:12.904038 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:12.904182 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.905201 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.906120 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:12.906853 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:12.906924 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:12.906994 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:12.907401 (+ 407us) server_negotiation.cc:299] Negotiation successful
0925 13:07:12.907449 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":854,"thread_start_us":66,"threads_started":1}
I20260925 13:07:12.908200 2076 tablet_service.cc:1467] Processing CreateTablet for tablet 8488fdc484b446eeb384192c91212826 (DEFAULT_TABLE table=test-table [id=8e98cf21cf4c472baadd0ef4cac489ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:12.908551 2076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8488fdc484b446eeb384192c91212826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:12.908597 2356 tablet_service.cc:1467] Processing CreateTablet for tablet 8488fdc484b446eeb384192c91212826 (DEFAULT_TABLE table=test-table [id=8e98cf21cf4c472baadd0ef4cac489ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:12.908959 2356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8488fdc484b446eeb384192c91212826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:12.911140 2216 tablet_service.cc:1467] Processing CreateTablet for tablet 8488fdc484b446eeb384192c91212826 (DEFAULT_TABLE table=test-table [id=8e98cf21cf4c472baadd0ef4cac489ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:12.911473 2216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8488fdc484b446eeb384192c91212826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:12.911866 2445 tablet_bootstrap.cc:492] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768: Bootstrap starting.
I20260925 13:07:12.912514 2445 tablet_bootstrap.cc:654] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:12.912726 2445 log.cc:824] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:12.913357 2445 tablet_bootstrap.cc:492] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768: No bootstrap required, opened a new log
I20260925 13:07:12.913399 2445 ts_tablet_manager.cc:1397] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:07:12.914317 2448 tablet_bootstrap.cc:492] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20: Bootstrap starting.
I20260925 13:07:12.914606 2446 tablet_bootstrap.cc:492] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e: Bootstrap starting.
I20260925 13:07:12.914992 2448 tablet_bootstrap.cc:654] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:12.915190 2446 tablet_bootstrap.cc:654] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:12.915324 2448 log.cc:824] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:12.915302 2445 raft_consensus.cc:348] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } }
I20260925 13:07:12.915473 2445 raft_consensus.cc:374] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:12.915494 2446 log.cc:824] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:12.915566 2445 raft_consensus.cc:729] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3f81f5c0ab48fb92be9039866d7768, State: Initialized, Role: FOLLOWER
I20260925 13:07:12.915727 2445 consensus_queue.cc:260] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } }
I20260925 13:07:12.916014 2448 tablet_bootstrap.cc:492] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20: No bootstrap required, opened a new log
I20260925 13:07:12.916010 2445 ts_tablet_manager.cc:1428] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 13:07:12.916049 2142 heartbeater.cc:503] Master 127.0.206.126:33717 was elected leader, sending a full tablet report...
I20260925 13:07:12.916079 2448 ts_tablet_manager.cc:1397] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:07:12.917726 2448 raft_consensus.cc:348] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } }
I20260925 13:07:12.917840 2448 raft_consensus.cc:374] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:12.917861 2448 raft_consensus.cc:729] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c3b4cc3dd14f8183a786373a29ff20, State: Initialized, Role: FOLLOWER
I20260925 13:07:12.917996 2448 consensus_queue.cc:260] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } }
I20260925 13:07:12.918246 2448 ts_tablet_manager.cc:1428] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:07:12.918315 2282 heartbeater.cc:503] Master 127.0.206.126:33717 was elected leader, sending a full tablet report...
I20260925 13:07:12.918632 2446 tablet_bootstrap.cc:492] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e: No bootstrap required, opened a new log
I20260925 13:07:12.918706 2446 ts_tablet_manager.cc:1397] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260925 13:07:12.920012 2446 raft_consensus.cc:348] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } }
I20260925 13:07:12.920147 2446 raft_consensus.cc:374] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:12.920190 2446 raft_consensus.cc:729] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bddd741031164c05a8801901c100295e, State: Initialized, Role: FOLLOWER
I20260925 13:07:12.920290 2446 consensus_queue.cc:260] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } }
I20260925 13:07:12.920521 2446 ts_tablet_manager.cc:1428] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260925 13:07:12.920594 2422 heartbeater.cc:503] Master 127.0.206.126:33717 was elected leader, sending a full tablet report...
W20260925 13:07:12.958376 2283 tablet.cc:2369] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:07:13.046763 2143 tablet.cc:2369] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:07:13.076975 2453 raft_consensus.cc:482] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:13.077131 2453 raft_consensus.cc:504] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } }
I20260925 13:07:13.077538 2453 leader_election.cc:288] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3c3b4cc3dd14f8183a786373a29ff20 (127.0.206.66:34789), 5a3f81f5c0ab48fb92be9039866d7768 (127.0.206.65:37717)
I20260925 13:07:13.080678 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.077866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:47575 (local address 127.0.206.66:34789)
0925 13:07:13.077932 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:13.077938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:13.078150 (+ 212us) server_negotiation.cc:407] Connection header received
0925 13:07:13.078177 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:13.078178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:13.078190 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:13.078264 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:13.078445 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.078995 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.079920 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.080295 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.080420 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.080430 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:13.080576 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 13:07:13.080611 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 13:07:13.080708 2454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.077744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:34789 (local address 127.0.206.67:47575)
0925 13:07:13.077986 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:13.077995 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:13.078043 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:13.078289 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:13.078290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:13.078292 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:13.078363 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:13.078365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.079058 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.079059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:13.079821 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:13.079823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.080421 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.080422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:13.080492 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.080495 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:07:13.080567 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 13:07:13.080605 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":12,"thread_start_us":124,"threads_started":1}
I20260925 13:07:13.080699 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.077820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:32891 (local address 127.0.206.65:37717)
0925 13:07:13.077860 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:13.077862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:13.077890 (+ 28us) server_negotiation.cc:407] Connection header received
0925 13:07:13.077932 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:13.077933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:13.077943 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:13.077997 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:13.078325 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.078908 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.079924 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.080374 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.080436 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.080444 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:13.080596 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 13:07:13.080632 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 13:07:13.080736 2315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.077744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:37717 (local address 127.0.206.67:32891)
0925 13:07:13.077797 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:13.077808 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:13.077855 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:13.078152 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:13.078153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:13.078155 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:13.078231 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:13.078233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.079008 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.079009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:13.079824 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:13.079826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.080434 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.080435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:13.080521 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.080523 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:07:13.080591 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 13:07:13.080629 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":54}
I20260925 13:07:13.081053 2096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8488fdc484b446eeb384192c91212826" candidate_uuid: "bddd741031164c05a8801901c100295e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3f81f5c0ab48fb92be9039866d7768" is_pre_election: true
I20260925 13:07:13.081152 2236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8488fdc484b446eeb384192c91212826" candidate_uuid: "bddd741031164c05a8801901c100295e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" is_pre_election: true
I20260925 13:07:13.081204 2096 raft_consensus.cc:2454] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bddd741031164c05a8801901c100295e in term 0.
I20260925 13:07:13.081275 2236 raft_consensus.cc:2454] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bddd741031164c05a8801901c100295e in term 0.
I20260925 13:07:13.081437 2310 leader_election.cc:302] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a3f81f5c0ab48fb92be9039866d7768, bddd741031164c05a8801901c100295e; no voters:
I20260925 13:07:13.081574 2453 raft_consensus.cc:2781] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:07:13.081641 2453 raft_consensus.cc:482] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:07:13.081669 2453 raft_consensus.cc:3037] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:13.082806 2453 raft_consensus.cc:504] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } }
I20260925 13:07:13.082957 2453 leader_election.cc:288] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [CANDIDATE]: Term 1 election: Requested vote from peers a3c3b4cc3dd14f8183a786373a29ff20 (127.0.206.66:34789), 5a3f81f5c0ab48fb92be9039866d7768 (127.0.206.65:37717)
I20260925 13:07:13.083077 2236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8488fdc484b446eeb384192c91212826" candidate_uuid: "bddd741031164c05a8801901c100295e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c3b4cc3dd14f8183a786373a29ff20"
I20260925 13:07:13.083105 2096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8488fdc484b446eeb384192c91212826" candidate_uuid: "bddd741031164c05a8801901c100295e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3f81f5c0ab48fb92be9039866d7768"
I20260925 13:07:13.083151 2236 raft_consensus.cc:3037] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:13.083196 2096 raft_consensus.cc:3037] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:13.083829 2096 raft_consensus.cc:2454] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bddd741031164c05a8801901c100295e in term 1.
I20260925 13:07:13.083832 2236 raft_consensus.cc:2454] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bddd741031164c05a8801901c100295e in term 1.
I20260925 13:07:13.084022 2307 leader_election.cc:302] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3c3b4cc3dd14f8183a786373a29ff20, bddd741031164c05a8801901c100295e; no voters:
I20260925 13:07:13.084146 2453 raft_consensus.cc:2781] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:13.084290 2453 raft_consensus.cc:686] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [term 1 LEADER]: Becoming Leader. State: Replica: bddd741031164c05a8801901c100295e, State: Running, Role: LEADER
I20260925 13:07:13.084403 2453 consensus_queue.cc:237] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } }
I20260925 13:07:13.085227 1939 catalog_manager.cc:5495] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e reported cstate change: term changed from 0 to 1, leader changed from <none> to bddd741031164c05a8801901c100295e (127.0.206.67). New cstate: current_term: 1 leader_uuid: "bddd741031164c05a8801901c100295e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } health_report { overall_health: UNKNOWN } } }
W20260925 13:07:13.107990 2423 tablet.cc:2369] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:07:13.131870 2443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.129660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46824 (local address 127.0.206.67:36335)
0925 13:07:13.129705 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:13.129706 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:13.129715 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:13.129815 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:13.129816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:13.129831 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:13.129891 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:13.130011 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.130462 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.131240 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.131413 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.131467 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.131618 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:13.131702 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:13.131757 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:13.131801 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 13:07:13.131816 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:07:13.133486 2236 raft_consensus.cc:1261] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20 [term 1 FOLLOWER]: Refusing update from remote peer bddd741031164c05a8801901c100295e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:13.133488 2096 raft_consensus.cc:1261] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768 [term 1 FOLLOWER]: Refusing update from remote peer bddd741031164c05a8801901c100295e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:13.133793 2453 consensus_queue.cc:1035] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:13.133931 2455 consensus_queue.cc:1035] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:13.136665 2458 mvcc.cc:205] Tried to move back new op lower bound from 7333239329312292864 to 7333239329114447872. Current Snapshot: MvccSnapshot[applied={T|T < 7333239329312292864}]
I20260925 13:07:13.137156 2459 mvcc.cc:205] Tried to move back new op lower bound from 7333239329312292864 to 7333239329114447872. Current Snapshot: MvccSnapshot[applied={T|T < 7333239329312292864 or (T in {7333239329312292864})}]
I20260925 13:07:13.140069 1939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:13.140192 1939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:13.141028 1939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569: Sending DeleteTablet for 3 replicas of tablet 8488fdc484b446eeb384192c91212826
I20260925 13:07:13.141394 2076 tablet_service.cc:1514] Processing DeleteTablet for tablet 8488fdc484b446eeb384192c91212826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:13 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:45996
I20260925 13:07:13.141443 2216 tablet_service.cc:1514] Processing DeleteTablet for tablet 8488fdc484b446eeb384192c91212826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:13 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:59332
I20260925 13:07:13.141685 2468 tablet_replica.cc:330] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20: stopping tablet replica
I20260925 13:07:13.141806 2468 raft_consensus.cc:2229] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:13.141790 2356 tablet_service.cc:1514] Processing DeleteTablet for tablet 8488fdc484b446eeb384192c91212826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:13 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:46818
I20260925 13:07:13.141957 2468 raft_consensus.cc:2258] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:13.142051 2467 tablet_replica.cc:330] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768: stopping tablet replica
I20260925 13:07:13.142077 1939 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:07:13.142148 2467 raft_consensus.cc:2229] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 13:07:13.142216 1939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:07:13.142235 2467 raft_consensus.cc:2258] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:13.142757 2468 ts_tablet_manager.cc:1905] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:13.143430 2076 tablet_service.cc:1467] Processing CreateTablet for tablet 405b775e3658494dba4f7a3e7380c166 (DEFAULT_TABLE table=test-table [id=cc6ac11efc6b46f08cef6bfc1d2a4f24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:13.143501 2216 tablet_service.cc:1467] Processing CreateTablet for tablet 405b775e3658494dba4f7a3e7380c166 (DEFAULT_TABLE table=test-table [id=cc6ac11efc6b46f08cef6bfc1d2a4f24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:13.143539 2076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 405b775e3658494dba4f7a3e7380c166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:13.143604 2216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 405b775e3658494dba4f7a3e7380c166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:13.144459 2468 ts_tablet_manager.cc:1918] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:07:13.144515 2468 log.cc:1196] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/8488fdc484b446eeb384192c91212826
I20260925 13:07:13.144811 2468 ts_tablet_manager.cc:1939] T 8488fdc484b446eeb384192c91212826 P a3c3b4cc3dd14f8183a786373a29ff20: Deleting consensus metadata
I20260925 13:07:13.144857 2448 tablet_bootstrap.cc:492] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20: Bootstrap starting.
I20260925 13:07:13.145138 2445 tablet_bootstrap.cc:492] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768: Bootstrap starting.
I20260925 13:07:13.145301 2448 tablet_bootstrap.cc:654] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:13.145414 2445 tablet_bootstrap.cc:654] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:13.145733 1923 catalog_manager.cc:4841] TS a3c3b4cc3dd14f8183a786373a29ff20 (127.0.206.66:34789): tablet 8488fdc484b446eeb384192c91212826 (table test-table [id=8e98cf21cf4c472baadd0ef4cac489ce]) successfully deleted
I20260925 13:07:13.145900 2469 tablet_replica.cc:330] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e: stopping tablet replica
I20260925 13:07:13.145984 2356 tablet_service.cc:1467] Processing CreateTablet for tablet 405b775e3658494dba4f7a3e7380c166 (DEFAULT_TABLE table=test-table [id=cc6ac11efc6b46f08cef6bfc1d2a4f24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:13.145994 2469 raft_consensus.cc:2229] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:07:13.146087 2356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 405b775e3658494dba4f7a3e7380c166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:13.146123 2469 raft_consensus.cc:2258] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:13.146138 2448 tablet_bootstrap.cc:492] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20: No bootstrap required, opened a new log
I20260925 13:07:13.146178 2448 ts_tablet_manager.cc:1397] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:13.146315 2448 raft_consensus.cc:348] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } }
I20260925 13:07:13.146374 2448 raft_consensus.cc:374] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:13.146394 2448 raft_consensus.cc:729] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c3b4cc3dd14f8183a786373a29ff20, State: Initialized, Role: FOLLOWER
I20260925 13:07:13.146438 2448 consensus_queue.cc:260] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } }
I20260925 13:07:13.146530 2448 ts_tablet_manager.cc:1428] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:13.146530 2469 ts_tablet_manager.cc:1905] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:13.147461 2446 tablet_bootstrap.cc:492] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e: Bootstrap starting.
I20260925 13:07:13.147883 2446 tablet_bootstrap.cc:654] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:13.148072 2469 ts_tablet_manager.cc:1918] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:07:13.148151 2469 log.cc:1196] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/8488fdc484b446eeb384192c91212826
I20260925 13:07:13.148419 2469 ts_tablet_manager.cc:1939] T 8488fdc484b446eeb384192c91212826 P bddd741031164c05a8801901c100295e: Deleting consensus metadata
I20260925 13:07:13.148589 2446 tablet_bootstrap.cc:492] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e: No bootstrap required, opened a new log
I20260925 13:07:13.148643 2446 ts_tablet_manager.cc:1397] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260925 13:07:13.148772 2446 raft_consensus.cc:348] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } }
I20260925 13:07:13.148792 1924 catalog_manager.cc:4841] TS bddd741031164c05a8801901c100295e (127.0.206.67:36335): tablet 8488fdc484b446eeb384192c91212826 (table test-table [id=8e98cf21cf4c472baadd0ef4cac489ce]) successfully deleted
I20260925 13:07:13.148838 2446 raft_consensus.cc:374] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:13.148859 2446 raft_consensus.cc:729] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bddd741031164c05a8801901c100295e, State: Initialized, Role: FOLLOWER
I20260925 13:07:13.148923 2446 consensus_queue.cc:260] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } }
I20260925 13:07:13.149019 2446 ts_tablet_manager.cc:1428] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:13.149225 2467 ts_tablet_manager.cc:1905] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:13.150996 2445 tablet_bootstrap.cc:492] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768: No bootstrap required, opened a new log
I20260925 13:07:13.151062 2445 ts_tablet_manager.cc:1397] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260925 13:07:13.151100 2467 ts_tablet_manager.cc:1918] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:07:13.151161 2467 log.cc:1196] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/wal/wals/8488fdc484b446eeb384192c91212826
I20260925 13:07:13.151204 2445 raft_consensus.cc:348] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } }
I20260925 13:07:13.151252 2445 raft_consensus.cc:374] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:13.151264 2445 raft_consensus.cc:729] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3f81f5c0ab48fb92be9039866d7768, State: Initialized, Role: FOLLOWER
I20260925 13:07:13.151317 2445 consensus_queue.cc:260] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } }
I20260925 13:07:13.151597 2467 ts_tablet_manager.cc:1939] T 8488fdc484b446eeb384192c91212826 P 5a3f81f5c0ab48fb92be9039866d7768: Deleting consensus metadata
I20260925 13:07:13.151600 2445 ts_tablet_manager.cc:1428] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 13:07:13.152012 1926 catalog_manager.cc:4841] TS 5a3f81f5c0ab48fb92be9039866d7768 (127.0.206.65:37717): tablet 8488fdc484b446eeb384192c91212826 (table test-table [id=8e98cf21cf4c472baadd0ef4cac489ce]) successfully deleted
I20260925 13:07:13.459937 2452 raft_consensus.cc:482] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:13.460043 2452 raft_consensus.cc:504] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } }
I20260925 13:07:13.460405 2452 leader_election.cc:288] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a3f81f5c0ab48fb92be9039866d7768 (127.0.206.65:37717), bddd741031164c05a8801901c100295e (127.0.206.67:36335)
I20260925 13:07:13.463254 2443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.460707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:33885 (local address 127.0.206.67:36335)
0925 13:07:13.460751 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:13.460753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:13.460820 (+ 67us) server_negotiation.cc:407] Connection header received
0925 13:07:13.460934 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:13.460935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:13.460947 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:13.460993 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:13.461155 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.461579 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.462516 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.462921 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.463005 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.463013 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:13.463148 (+ 135us) server_negotiation.cc:299] Negotiation successful
0925 13:07:13.463185 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:07:13.463274 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.460704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:44655 (local address 127.0.206.65:37717)
0925 13:07:13.460763 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:13.460766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:13.460779 (+ 13us) server_negotiation.cc:407] Connection header received
0925 13:07:13.460849 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:13.460850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:13.460860 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:13.460916 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:13.461116 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.461580 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.462517 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.462916 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.462990 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.462999 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:13.463162 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 13:07:13.463195 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 13:07:13.463289 2473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.460581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:37717 (local address 127.0.206.66:44655)
0925 13:07:13.460705 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:13.460717 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:13.460778 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:13.460960 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:13.460961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:13.460963 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:13.461045 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:13.461048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.461665 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.461666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:13.462425 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:13.462427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.462991 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.462992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:13.463067 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.463071 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:07:13.463154 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 13:07:13.463194 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":43,"thread_start_us":60,"threads_started":1}
I20260925 13:07:13.463290 2474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.460605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:36335 (local address 127.0.206.66:33885)
0925 13:07:13.460759 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:13.460771 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:13.460837 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:13.461011 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:13.461012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:13.461013 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:13.461064 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:13.461065 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.461655 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.461656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:13.462428 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:13.462429 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.463015 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.463016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:13.463077 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.463080 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:07:13.463146 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 13:07:13.463189 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":2,"thread_start_us":53,"threads_started":1}
I20260925 13:07:13.463533 2096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "405b775e3658494dba4f7a3e7380c166" candidate_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3f81f5c0ab48fb92be9039866d7768" is_pre_election: true
I20260925 13:07:13.463620 2096 raft_consensus.cc:2454] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3c3b4cc3dd14f8183a786373a29ff20 in term 0.
I20260925 13:07:13.463656 2376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "405b775e3658494dba4f7a3e7380c166" candidate_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bddd741031164c05a8801901c100295e" is_pre_election: true
I20260925 13:07:13.463784 2376 raft_consensus.cc:2454] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3c3b4cc3dd14f8183a786373a29ff20 in term 0.
I20260925 13:07:13.463798 2170 leader_election.cc:302] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a3f81f5c0ab48fb92be9039866d7768, a3c3b4cc3dd14f8183a786373a29ff20; no voters:
I20260925 13:07:13.463965 2452 raft_consensus.cc:2781] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:07:13.463996 2452 raft_consensus.cc:482] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:07:13.464035 2452 raft_consensus.cc:3037] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:13.464720 2452 raft_consensus.cc:504] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } }
I20260925 13:07:13.464859 2452 leader_election.cc:288] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [CANDIDATE]: Term 1 election: Requested vote from peers 5a3f81f5c0ab48fb92be9039866d7768 (127.0.206.65:37717), bddd741031164c05a8801901c100295e (127.0.206.67:36335)
I20260925 13:07:13.465020 2096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "405b775e3658494dba4f7a3e7380c166" candidate_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3f81f5c0ab48fb92be9039866d7768"
I20260925 13:07:13.465059 2376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "405b775e3658494dba4f7a3e7380c166" candidate_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bddd741031164c05a8801901c100295e"
I20260925 13:07:13.465106 2096 raft_consensus.cc:3037] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:13.465139 2376 raft_consensus.cc:3037] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:13.465801 2376 raft_consensus.cc:2454] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3c3b4cc3dd14f8183a786373a29ff20 in term 1.
I20260925 13:07:13.465965 2096 raft_consensus.cc:2454] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3c3b4cc3dd14f8183a786373a29ff20 in term 1.
I20260925 13:07:13.465972 2168 leader_election.cc:302] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3c3b4cc3dd14f8183a786373a29ff20, bddd741031164c05a8801901c100295e; no voters:
I20260925 13:07:13.466079 2452 raft_consensus.cc:2781] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:13.466192 2452 raft_consensus.cc:686] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [term 1 LEADER]: Becoming Leader. State: Replica: a3c3b4cc3dd14f8183a786373a29ff20, State: Running, Role: LEADER
I20260925 13:07:13.466257 2452 consensus_queue.cc:237] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } }
I20260925 13:07:13.466894 1938 catalog_manager.cc:5495] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3c3b4cc3dd14f8183a786373a29ff20 (127.0.206.66). New cstate: current_term: 1 leader_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } health_report { overall_health: UNKNOWN } } }
Sep 25 13:07:13 dist-test-slave-chkf krb5kdc[1894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341632, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:13.521889 2482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.515262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33717 (local address 127.0.0.1:34976)
0925 13:07:13.515465 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:13.515476 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:13.515555 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:13.515705 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:13.515706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:13.515998 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:13.516039 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:13.516041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.516648 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.516649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:13.517136 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:13.517137 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.517473 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.517474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:13.517521 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.518044 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:13.518048 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:13.519383 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:13.520778 (+ 1395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:13.520785 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:13.520787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:13.520972 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:13.521170 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:13.521171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:13.521171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:13.521229 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:13.521467 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:13.521474 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:13.521515 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 13:07:13.521598 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 13:07:13.521721 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":4,"thread_start_us":111,"threads_started":1}
I20260925 13:07:13.521903 2483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.515339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34976 (local address 127.0.206.126:33717)
0925 13:07:13.515452 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:13.515454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:13.515538 (+ 84us) server_negotiation.cc:407] Connection header received
0925 13:07:13.515611 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:13.515612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:13.515623 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:13.515703 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:13.516125 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.516576 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.517225 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.517399 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.517474 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.519500 (+ 2026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:13.519501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:13.519501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:13.519507 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:13.520689 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:13.521043 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:13.521045 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:13.521045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:13.521105 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:13.521299 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:13.521300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:13.521300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:13.521398 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:13.521470 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:13.521621 (+ 151us) server_negotiation.cc:299] Negotiation successful
0925 13:07:13.521769 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":93,"thread_start_us":65,"threads_started":1}
I20260925 13:07:13.523828 1938 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:34976:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 13:07:13.524044 1938 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:07:13.525430 2076 tablet_service.cc:1467] Processing CreateTablet for tablet 50bba0c229f44b1fb5ee80889db21c67 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cb0f3916ee4b47599cf6971ae4f59f63]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 13:07:13.525444 2216 tablet_service.cc:1467] Processing CreateTablet for tablet 50bba0c229f44b1fb5ee80889db21c67 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cb0f3916ee4b47599cf6971ae4f59f63]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 13:07:13.525473 2356 tablet_service.cc:1467] Processing CreateTablet for tablet 50bba0c229f44b1fb5ee80889db21c67 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cb0f3916ee4b47599cf6971ae4f59f63]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 13:07:13.525583 2356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50bba0c229f44b1fb5ee80889db21c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:13.525583 2216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50bba0c229f44b1fb5ee80889db21c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:13.525599 2076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50bba0c229f44b1fb5ee80889db21c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:13.526710 2445 tablet_bootstrap.cc:492] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768: Bootstrap starting.
I20260925 13:07:13.526968 2446 tablet_bootstrap.cc:492] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e: Bootstrap starting.
I20260925 13:07:13.527176 2445 tablet_bootstrap.cc:654] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:13.527290 2446 tablet_bootstrap.cc:654] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:13.527554 2448 tablet_bootstrap.cc:492] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20: Bootstrap starting.
I20260925 13:07:13.527885 2445 tablet_bootstrap.cc:492] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768: No bootstrap required, opened a new log
I20260925 13:07:13.527930 2445 ts_tablet_manager.cc:1397] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:13.528004 2446 tablet_bootstrap.cc:492] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e: No bootstrap required, opened a new log
I20260925 13:07:13.528002 2448 tablet_bootstrap.cc:654] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:13.528039 2446 ts_tablet_manager.cc:1397] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260925 13:07:13.528075 2445 raft_consensus.cc:348] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } }
I20260925 13:07:13.528136 2446 raft_consensus.cc:348] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } }
I20260925 13:07:13.528139 2445 raft_consensus.cc:374] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:13.528178 2446 raft_consensus.cc:374] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:13.528192 2446 raft_consensus.cc:729] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bddd741031164c05a8801901c100295e, State: Initialized, Role: FOLLOWER
I20260925 13:07:13.528184 2445 raft_consensus.cc:729] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3f81f5c0ab48fb92be9039866d7768, State: Initialized, Role: FOLLOWER
I20260925 13:07:13.528251 2446 consensus_queue.cc:260] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } }
I20260925 13:07:13.528265 2445 consensus_queue.cc:260] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } }
I20260925 13:07:13.528340 2446 ts_tablet_manager.cc:1428] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:13.528383 2445 ts_tablet_manager.cc:1428] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:13.528352 2455 tablet_replica.cc:439] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } }
I20260925 13:07:13.528434 2455 tablet_replica.cc:442] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:13.528504 2451 tablet_replica.cc:439] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } }
I20260925 13:07:13.528596 2451 tablet_replica.cc:442] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:13.528915 2448 tablet_bootstrap.cc:492] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20: No bootstrap required, opened a new log
I20260925 13:07:13.528965 2448 ts_tablet_manager.cc:1397] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:13.529134 2448 raft_consensus.cc:348] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } }
I20260925 13:07:13.529188 2448 raft_consensus.cc:374] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:13.529201 2448 raft_consensus.cc:729] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c3b4cc3dd14f8183a786373a29ff20, State: Initialized, Role: FOLLOWER
I20260925 13:07:13.529253 2448 consensus_queue.cc:260] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } }
I20260925 13:07:13.529345 2448 ts_tablet_manager.cc:1428] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:13.529381 2452 tablet_replica.cc:439] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } }
I20260925 13:07:13.529448 2452 tablet_replica.cc:442] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:13.613658 2455 raft_consensus.cc:482] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:13.613747 2455 raft_consensus.cc:504] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } }
I20260925 13:07:13.613888 2455 leader_election.cc:288] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a3f81f5c0ab48fb92be9039866d7768 (127.0.206.65:37717), a3c3b4cc3dd14f8183a786373a29ff20 (127.0.206.66:34789)
I20260925 13:07:13.614074 2096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50bba0c229f44b1fb5ee80889db21c67" candidate_uuid: "bddd741031164c05a8801901c100295e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3f81f5c0ab48fb92be9039866d7768" is_pre_election: true
I20260925 13:07:13.614156 2096 raft_consensus.cc:2454] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bddd741031164c05a8801901c100295e in term 0.
I20260925 13:07:13.614128 2236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50bba0c229f44b1fb5ee80889db21c67" candidate_uuid: "bddd741031164c05a8801901c100295e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" is_pre_election: true
I20260925 13:07:13.614239 2236 raft_consensus.cc:2454] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bddd741031164c05a8801901c100295e in term 0.
I20260925 13:07:13.614293 2310 leader_election.cc:302] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a3f81f5c0ab48fb92be9039866d7768, bddd741031164c05a8801901c100295e; no voters:
I20260925 13:07:13.614406 2455 raft_consensus.cc:2781] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:07:13.614451 2455 raft_consensus.cc:482] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:07:13.614478 2455 raft_consensus.cc:3037] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:13.615182 2455 raft_consensus.cc:504] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } }
I20260925 13:07:13.615329 2455 leader_election.cc:288] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [CANDIDATE]: Term 1 election: Requested vote from peers 5a3f81f5c0ab48fb92be9039866d7768 (127.0.206.65:37717), a3c3b4cc3dd14f8183a786373a29ff20 (127.0.206.66:34789)
I20260925 13:07:13.615463 2096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50bba0c229f44b1fb5ee80889db21c67" candidate_uuid: "bddd741031164c05a8801901c100295e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3f81f5c0ab48fb92be9039866d7768"
I20260925 13:07:13.615533 2096 raft_consensus.cc:3037] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:13.615494 2236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50bba0c229f44b1fb5ee80889db21c67" candidate_uuid: "bddd741031164c05a8801901c100295e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3c3b4cc3dd14f8183a786373a29ff20"
I20260925 13:07:13.615579 2236 raft_consensus.cc:3037] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:13.616078 2096 raft_consensus.cc:2454] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bddd741031164c05a8801901c100295e in term 1.
I20260925 13:07:13.616156 2236 raft_consensus.cc:2454] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bddd741031164c05a8801901c100295e in term 1.
I20260925 13:07:13.616248 2310 leader_election.cc:302] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a3f81f5c0ab48fb92be9039866d7768, bddd741031164c05a8801901c100295e; no voters:
I20260925 13:07:13.616364 2455 raft_consensus.cc:2781] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:13.616441 2455 raft_consensus.cc:686] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [term 1 LEADER]: Becoming Leader. State: Replica: bddd741031164c05a8801901c100295e, State: Running, Role: LEADER
I20260925 13:07:13.616493 2455 consensus_queue.cc:237] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } }
I20260925 13:07:13.616772 2453 tablet_replica.cc:439] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e: TxnStatusTablet state changed. Reason: New leader bddd741031164c05a8801901c100295e. Latest consensus state: current_term: 1 leader_uuid: "bddd741031164c05a8801901c100295e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } }
I20260925 13:07:13.616859 2453 tablet_replica.cc:442] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e: This TxnStatusTablet replica's current role is: LEADER
I20260925 13:07:13.617017 2488 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 13:07:13.617035 1939 catalog_manager.cc:5495] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e reported cstate change: term changed from 0 to 1, leader changed from <none> to bddd741031164c05a8801901c100295e (127.0.206.67). New cstate: current_term: 1 leader_uuid: "bddd741031164c05a8801901c100295e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } health_report { overall_health: HEALTHY } } }
I20260925 13:07:13.617138 2488 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 13:07:13.622951 2443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.620448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46832 (local address 127.0.206.67:36335)
0925 13:07:13.620496 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:13.620498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:13.620506 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:13.620549 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:13.620550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:13.620557 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:13.620607 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:13.620721 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.621166 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.622250 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.622440 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.622497 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.622677 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:13.622743 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:13.622797 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:13.622881 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 13:07:13.622900 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 13:07:13.623024 2482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.620345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:36335 (local address 127.0.0.1:46832)
0925 13:07:13.620421 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:13.620432 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:13.620484 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:13.620614 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:13.620615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:13.620616 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 13:07:13.620662 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:13.620665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.621238 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.621239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:13.622149 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:13.622152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.622517 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.622518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:13.622577 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.622593 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 13:07:13.622812 (+ 219us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 13:07:13.622820 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 13:07:13.622877 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 13:07:13.622932 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 13:07:13.623958 2096 raft_consensus.cc:1261] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768 [term 1 FOLLOWER]: Refusing update from remote peer bddd741031164c05a8801901c100295e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:13.623955 2236 raft_consensus.cc:1261] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20 [term 1 FOLLOWER]: Refusing update from remote peer bddd741031164c05a8801901c100295e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:13.624147 2455 consensus_queue.cc:1035] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:13.624255 2453 consensus_queue.cc:1035] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:13.624754 2452 tablet_replica.cc:439] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20: TxnStatusTablet state changed. Reason: New leader bddd741031164c05a8801901c100295e. Latest consensus state: current_term: 1 leader_uuid: "bddd741031164c05a8801901c100295e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } }
I20260925 13:07:13.624866 2452 tablet_replica.cc:442] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:13.625043 2451 tablet_replica.cc:439] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768: TxnStatusTablet state changed. Reason: New leader bddd741031164c05a8801901c100295e. Latest consensus state: current_term: 1 leader_uuid: "bddd741031164c05a8801901c100295e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } }
I20260925 13:07:13.625121 2451 tablet_replica.cc:442] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:13.625437 2455 tablet_replica.cc:439] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bddd741031164c05a8801901c100295e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } }
I20260925 13:07:13.625515 2455 tablet_replica.cc:442] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e: This TxnStatusTablet replica's current role is: LEADER
I20260925 13:07:13.625764 2460 mvcc.cc:205] Tried to move back new op lower bound from 7333239331322445824 to 7333239331293601792. Current Snapshot: MvccSnapshot[applied={T|T < 7333239331322445824 or (T in {7333239331322445824})}]
I20260925 13:07:13.626013 2455 tablet_replica.cc:439] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bddd741031164c05a8801901c100295e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } }
I20260925 13:07:13.626053 2451 tablet_replica.cc:439] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bddd741031164c05a8801901c100295e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } }
I20260925 13:07:13.626098 2455 tablet_replica.cc:442] T 50bba0c229f44b1fb5ee80889db21c67 P bddd741031164c05a8801901c100295e: This TxnStatusTablet replica's current role is: LEADER
I20260925 13:07:13.626118 2451 tablet_replica.cc:442] T 50bba0c229f44b1fb5ee80889db21c67 P 5a3f81f5c0ab48fb92be9039866d7768: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:13.626433 2452 tablet_replica.cc:439] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bddd741031164c05a8801901c100295e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 } } peers { permanent_uuid: "a3c3b4cc3dd14f8183a786373a29ff20" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34789 } } peers { permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 } } }
I20260925 13:07:13.626523 2452 tablet_replica.cc:442] T 50bba0c229f44b1fb5ee80889db21c67 P a3c3b4cc3dd14f8183a786373a29ff20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:13.630119 2493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.627697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59338 (local address 127.0.206.66:34789)
0925 13:07:13.627869 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:13.627871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:13.627877 (+ 6us) server_negotiation.cc:407] Connection header received
0925 13:07:13.627918 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:13.627919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:13.627935 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:13.628019 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:13.628115 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.628578 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.629298 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.629513 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.629611 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.629776 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:13.629879 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:13.629961 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:13.630002 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 13:07:13.630021 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":109,"threads_started":1}
I20260925 13:07:13.634780 2443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.632411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:49729 (local address 127.0.206.67:36335)
0925 13:07:13.632447 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:13.632448 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:13.632623 (+ 175us) server_negotiation.cc:407] Connection header received
0925 13:07:13.632691 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:13.632691 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:13.632701 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:13.632749 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:13.632872 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.633287 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.634050 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.634260 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.634331 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.634507 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:13.634536 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:13.634606 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:13.634665 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 13:07:13.634682 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 13:07:13.634809 2496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.632364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:36335 (local address 127.0.206.66:49729)
0925 13:07:13.632555 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:13.632563 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:13.632636 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:13.632753 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:13.632753 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:13.632754 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 13:07:13.632807 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:13.632809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.633379 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.633380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:13.633958 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:13.633960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.634346 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.634347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:13.634418 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.634429 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 13:07:13.634606 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 13:07:13.634608 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:07:13.634661 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 13:07:13.634694 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":64,"threads_started":1}
I20260925 13:07:13.636709 2096 raft_consensus.cc:1261] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768 [term 1 FOLLOWER]: Refusing update from remote peer a3c3b4cc3dd14f8183a786373a29ff20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:13.636842 2376 raft_consensus.cc:1261] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e [term 1 FOLLOWER]: Refusing update from remote peer a3c3b4cc3dd14f8183a786373a29ff20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:13.637099 2475 consensus_queue.cc:1035] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a3f81f5c0ab48fb92be9039866d7768" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:13.637400 2475 consensus_queue.cc:1035] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bddd741031164c05a8801901c100295e" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:13.644820 2500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.642440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:34789 (local address 127.0.206.67:43583)
0925 13:07:13.642614 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:13.642625 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:13.642678 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:13.642823 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:13.642824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:13.642825 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 13:07:13.642862 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:13.642863 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.643429 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.643430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:13.644034 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:13.644036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.644458 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.644459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:13.644520 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.644539 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 13:07:13.644708 (+ 169us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 13:07:13.644709 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 13:07:13.644741 (+ 32us) client_negotiation.cc:240] Negotiation successful
0925 13:07:13.644771 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":77,"threads_started":1}
I20260925 13:07:13.644822 2493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.642600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:43583 (local address 127.0.206.66:34789)
0925 13:07:13.642630 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:13.642632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:13.642664 (+ 32us) server_negotiation.cc:407] Connection header received
0925 13:07:13.642763 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:13.642764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:13.642772 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:13.642818 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:13.642916 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.643353 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.644113 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.644394 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.644468 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.644620 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:13.644649 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:13.644699 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:13.644749 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 13:07:13.644765 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 13:07:13.654114 1938 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:13.654207 1938 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:13.654796 1938 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f434f38b0b0a48749433f3a4e1e2f569: Sending DeleteTablet for 3 replicas of tablet 405b775e3658494dba4f7a3e7380c166
I20260925 13:07:13.655025 2076 tablet_service.cc:1514] Processing DeleteTablet for tablet 405b775e3658494dba4f7a3e7380c166 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:13 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:45996
I20260925 13:07:13.655160 2216 tablet_service.cc:1514] Processing DeleteTablet for tablet 405b775e3658494dba4f7a3e7380c166 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:13 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:59332
I20260925 13:07:13.655360 2506 tablet_replica.cc:330] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20: stopping tablet replica
I20260925 13:07:13.655432 2506 raft_consensus.cc:2229] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:07:13.655531 2506 raft_consensus.cc:2258] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:13.655601 2356 tablet_service.cc:1514] Processing DeleteTablet for tablet 405b775e3658494dba4f7a3e7380c166 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:13 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:46818
I20260925 13:07:13.655721 2507 tablet_replica.cc:330] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e: stopping tablet replica
I20260925 13:07:13.655798 2507 raft_consensus.cc:2229] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:13.655862 2507 raft_consensus.cc:2258] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:13.656595 2506 ts_tablet_manager.cc:1905] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:13.657862 2507 ts_tablet_manager.cc:1905] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:13.658849 2506 ts_tablet_manager.cc:1918] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 13:07:13.658957 2506 log.cc:1196] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/405b775e3658494dba4f7a3e7380c166
I20260925 13:07:13.659235 2507 ts_tablet_manager.cc:1918] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 13:07:13.659260 2506 ts_tablet_manager.cc:1939] T 405b775e3658494dba4f7a3e7380c166 P a3c3b4cc3dd14f8183a786373a29ff20: Deleting consensus metadata
I20260925 13:07:13.659381 2507 log.cc:1196] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/405b775e3658494dba4f7a3e7380c166
I20260925 13:07:13.659497 2505 tablet_replica.cc:330] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768: stopping tablet replica
I20260925 13:07:13.659541 1923 catalog_manager.cc:4841] TS a3c3b4cc3dd14f8183a786373a29ff20 (127.0.206.66:34789): tablet 405b775e3658494dba4f7a3e7380c166 (table test-table [id=cc6ac11efc6b46f08cef6bfc1d2a4f24]) successfully deleted
I20260925 13:07:13.659550 2505 raft_consensus.cc:2229] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:13.659597 2505 raft_consensus.cc:2258] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:13.659832 2505 ts_tablet_manager.cc:1905] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:13.659953 2507 ts_tablet_manager.cc:1939] T 405b775e3658494dba4f7a3e7380c166 P bddd741031164c05a8801901c100295e: Deleting consensus metadata
I20260925 13:07:13.660282 1924 catalog_manager.cc:4841] TS bddd741031164c05a8801901c100295e (127.0.206.67:36335): tablet 405b775e3658494dba4f7a3e7380c166 (table test-table [id=cc6ac11efc6b46f08cef6bfc1d2a4f24]) successfully deleted
I20260925 13:07:13.661193 2505 ts_tablet_manager.cc:1918] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 13:07:13.661269 2505 log.cc:1196] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790341627249256-825-0/minicluster-data/ts-0/wal/wals/405b775e3658494dba4f7a3e7380c166
I20260925 13:07:13.661474 2505 ts_tablet_manager.cc:1939] T 405b775e3658494dba4f7a3e7380c166 P 5a3f81f5c0ab48fb92be9039866d7768: Deleting consensus metadata
I20260925 13:07:13.661818 1926 catalog_manager.cc:4841] TS 5a3f81f5c0ab48fb92be9039866d7768 (127.0.206.65:37717): tablet 405b775e3658494dba4f7a3e7380c166 (table test-table [id=cc6ac11efc6b46f08cef6bfc1d2a4f24]) successfully deleted
Sep 25 13:07:13 dist-test-slave-chkf krb5kdc[1894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341632, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260925 13:07:13.669790 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.656427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46012 (local address 127.0.206.65:37717)
0925 13:07:13.656466 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:13.656467 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:13.661860 (+ 5393us) server_negotiation.cc:407] Connection header received
0925 13:07:13.661965 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:13.661966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:13.661976 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:13.662050 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:13.662588 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.663106 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.663906 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.664090 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.664168 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.666831 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:13.666840 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:13.666845 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:13.666852 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:13.668542 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:13.668923 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:13.668926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:13.668927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:13.668999 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:13.669243 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:13.669244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:13.669244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:13.669376 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:13.669525 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:13.669588 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 13:07:13.669662 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 13:07:13.670397 2136 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46012
I20260925 13:07:13.680894 2483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:13.671230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34980 (local address 127.0.206.126:33717)
0925 13:07:13.671281 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:13.671283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:13.674876 (+ 3593us) server_negotiation.cc:407] Connection header received
0925 13:07:13.674946 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:13.674947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:13.674959 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:13.675013 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:13.675485 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.675968 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.676675 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:13.676884 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:13.676916 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:13.678180 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:13.678182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:13.678182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:13.678189 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:13.679695 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:13.680087 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:13.680088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:13.680089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:13.680143 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:13.680356 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:13.680357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:13.680357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:13.680472 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:13.680582 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:13.680702 (+ 120us) server_negotiation.cc:299] Negotiation successful
0925 13:07:13.680783 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260925 13:07:13.681085 1938 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34980
I20260925 13:07:13.681672 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 2011
I20260925 13:07:13.687995 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 2151
I20260925 13:07:13.693389 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 2291
I20260925 13:07:13.697916 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 1907
2026-09-25T13:07:13Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2592 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:07:13 dist-test-slave-chkf krb5kdc[2514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:07:13 dist-test-slave-chkf krb5kdc[2514](info): set up 2 sockets
Sep 25 13:07:13 dist-test-slave-chkf krb5kdc[2514](info): commencing operation
krb5kdc: starting...
W20260925 13:07:14.714293 825 mini_kdc.cc:121] Time spent starting KDC: real 0.996s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:07:14 dist-test-slave-chkf krb5kdc[2514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:07:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:07:14.822595 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:46693
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:34905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:46693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:14.899588 2527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:14.899765 2527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:14.899788 2527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:14.900913 2527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:14.900954 2527 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 13:07:14.900969 2527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:14.901018 2527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:14.901036 2527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:14.901058 2527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:07:14.902460 2527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:34905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:46693
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:46693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:14.902691 2527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:14.902926 2527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:14.907580 2533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:14.907559 2532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:14.907546 2535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:14.907580 2527 server_base.cc:1034] running on GCE node
I20260925 13:07:14.908018 2527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:14.908288 2527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:14.909423 2527 hybrid_clock.cc:648] HybridClock initialized: now 1790341634909408 us; error 31 us; skew 500 ppm
Sep 25 13:07:14 dist-test-slave-chkf krb5kdc[2514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341634, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:14.911880 2527 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:07:14.912261 2527 webserver.cc:460] Webserver started at http://127.0.206.126:35753/ using document root <none> and password file <none>
I20260925 13:07:14.912474 2527 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:14.912539 2527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:14.912685 2527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:14.913621 2527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "29e5d573cd3246bd9622d7574658e3f9"
format_stamp: "Formatted at 2026-09-25 13:07:14 on dist-test-slave-chkf"
server_key: "830eb3b189b8e6153872a25e10198af4"
server_key_iv: "26496a1bb1291fb1e70b130023004e9a"
server_key_version: "encryptionkey@0"
I20260925 13:07:14.913965 2527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "29e5d573cd3246bd9622d7574658e3f9"
format_stamp: "Formatted at 2026-09-25 13:07:14 on dist-test-slave-chkf"
server_key: "830eb3b189b8e6153872a25e10198af4"
server_key_iv: "26496a1bb1291fb1e70b130023004e9a"
server_key_version: "encryptionkey@0"
I20260925 13:07:14.915237 2527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:07:14.915879 2542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:14.916036 2527 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:14.916134 2527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "29e5d573cd3246bd9622d7574658e3f9"
format_stamp: "Formatted at 2026-09-25 13:07:14 on dist-test-slave-chkf"
server_key: "830eb3b189b8e6153872a25e10198af4"
server_key_iv: "26496a1bb1291fb1e70b130023004e9a"
server_key_version: "encryptionkey@0"
I20260925 13:07:14.916201 2527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:14.930660 2527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:14.932873 2527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:14.933020 2527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:14.937363 2527 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:46693
I20260925 13:07:14.937431 2604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:46693 every 8 connection(s)
I20260925 13:07:14.937734 2527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:14.938372 2605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:14.940464 2605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9: Bootstrap starting.
I20260925 13:07:14.941010 2605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:14.941286 2605 log.cc:824] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:14.942011 2605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9: No bootstrap required, opened a new log
I20260925 13:07:14.944139 2605 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e5d573cd3246bd9622d7574658e3f9" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46693 } }
I20260925 13:07:14.944276 2605 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:14.944299 2605 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29e5d573cd3246bd9622d7574658e3f9, State: Initialized, Role: FOLLOWER
I20260925 13:07:14.944456 2605 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e5d573cd3246bd9622d7574658e3f9" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46693 } }
I20260925 13:07:14.944525 2605 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:14.944566 2605 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:14.944614 2605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:14.945104 2605 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e5d573cd3246bd9622d7574658e3f9" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46693 } }
I20260925 13:07:14.945185 2605 leader_election.cc:302] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29e5d573cd3246bd9622d7574658e3f9; no voters:
I20260925 13:07:14.945350 2605 leader_election.cc:288] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:07:14.945443 2608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:14.945573 2605 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:14.945634 2608 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9 [term 1 LEADER]: Becoming Leader. State: Replica: 29e5d573cd3246bd9622d7574658e3f9, State: Running, Role: LEADER
I20260925 13:07:14.945799 2608 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e5d573cd3246bd9622d7574658e3f9" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46693 } }
I20260925 13:07:14.946187 2609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29e5d573cd3246bd9622d7574658e3f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e5d573cd3246bd9622d7574658e3f9" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46693 } } }
I20260925 13:07:14.946264 2609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:14.946558 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 2527
I20260925 13:07:14.946646 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:07:14.946172 2610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29e5d573cd3246bd9622d7574658e3f9. Latest consensus state: current_term: 1 leader_uuid: "29e5d573cd3246bd9622d7574658e3f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e5d573cd3246bd9622d7574658e3f9" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46693 } } }
I20260925 13:07:14.946745 2617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:14.946734 2610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:14.946928 825 external_mini_cluster.cc:1428] Setting key a924999ba392cc3f125888743a33a0de
I20260925 13:07:14.947356 2617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:07:14.950055 2617 catalog_manager.cc:1347] Generated new cluster ID: f9ed287514f04cd9a9e0a4279581b2dc
I20260925 13:07:14.950107 2617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 13:07:14 dist-test-slave-chkf krb5kdc[2514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:14.956693 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:14.947352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43754 (local address 127.0.206.126:46693)
0925 13:07:14.947532 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:14.947541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:14.947569 (+ 28us) server_negotiation.cc:407] Connection header received
0925 13:07:14.947673 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:14.947679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:14.947718 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:14.947863 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:14.948559 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:14.949918 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:14.950743 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:14.951189 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:14.951326 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:14.953434 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:14.953439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:14.953440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:14.953453 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:14.955152 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:14.955540 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:14.955542 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:14.955543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:14.955637 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:14.955841 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:14.955842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:14.955842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:14.956057 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:14.956212 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:14.956327 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 13:07:14.956546 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":78,"threads_started":1}
I20260925 13:07:14.960383 2617 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:07:14.960855 2617 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:14.964270 2617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9: Generated new TSK 0
I20260925 13:07:14.964423 2617 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260925 13:07:14.993373 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:46693
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:34905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:15.095803 2631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:15.096024 2631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:15.096063 2631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:15.097182 2631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:15.097251 2631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:15.097285 2631 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 13:07:15.097316 2631 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 13:07:15.097348 2631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:07:15.098798 2631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:34905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.206.126:46693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:15.099044 2631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:15.099256 2631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:15.104187 2637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:15.104206 2639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:15.104390 2631 server_base.cc:1034] running on GCE node
W20260925 13:07:15.104511 2636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:15.104696 2631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:15.104944 2631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:15.106094 2631 hybrid_clock.cc:648] HybridClock initialized: now 1790341635106073 us; error 38 us; skew 500 ppm
Sep 25 13:07:15 dist-test-slave-chkf krb5kdc[2514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341635, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:15.108675 2631 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:07:15.109122 2631 webserver.cc:460] Webserver started at http://127.0.206.65:41629/ using document root <none> and password file <none>
I20260925 13:07:15.109351 2631 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:15.109417 2631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:15.109571 2631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:15.110661 2631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "4286e8349aa04b22bd638c383898ae9f"
format_stamp: "Formatted at 2026-09-25 13:07:15 on dist-test-slave-chkf"
server_key: "b03f50f59126bd5c5cb643559c088373"
server_key_iv: "b78c4769bbd1b4f256dedf7299ca7f73"
server_key_version: "encryptionkey@0"
I20260925 13:07:15.111095 2631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "4286e8349aa04b22bd638c383898ae9f"
format_stamp: "Formatted at 2026-09-25 13:07:15 on dist-test-slave-chkf"
server_key: "b03f50f59126bd5c5cb643559c088373"
server_key_iv: "b78c4769bbd1b4f256dedf7299ca7f73"
server_key_version: "encryptionkey@0"
I20260925 13:07:15.112443 2631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:15.113358 2646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:15.113569 2631 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:15.113658 2631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "4286e8349aa04b22bd638c383898ae9f"
format_stamp: "Formatted at 2026-09-25 13:07:15 on dist-test-slave-chkf"
server_key: "b03f50f59126bd5c5cb643559c088373"
server_key_iv: "b78c4769bbd1b4f256dedf7299ca7f73"
server_key_version: "encryptionkey@0"
I20260925 13:07:15.113744 2631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:15.131996 2631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:15.134354 2631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:15.134518 2631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:15.135143 2631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:15.135202 2631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:15.135233 2631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:15.135264 2631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 13:07:15 dist-test-slave-chkf krb5kdc[2514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341635, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:15.142560 2631 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:36595
I20260925 13:07:15.142664 2761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:36595 every 8 connection(s)
I20260925 13:07:15.142993 2631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:07:15.145038 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.135618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:35005 (local address 127.0.206.126:46693)
0925 13:07:15.135695 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.135697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.135971 (+ 274us) server_negotiation.cc:407] Connection header received
0925 13:07:15.136701 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.136702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.136712 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.136791 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:15.137709 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.138549 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.139623 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.139860 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.139947 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.142074 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:15.142076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:15.142076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:15.142085 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:15.143779 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:15.144169 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:15.144170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:15.144170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:15.144216 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:15.144486 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:15.144486 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:15.144487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:15.144582 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:15.144632 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.144903 (+ 271us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.144976 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 13:07:15.145203 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.135604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:46693 (local address 127.0.206.65:35005)
0925 13:07:15.135820 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.135870 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.136588 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.136895 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.136901 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.137301 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:15.137614 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.137624 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.138647 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.138648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.139524 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.139527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.139970 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.139972 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.140098 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.140601 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:15.140617 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:15.141968 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:15.143857 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:15.143862 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:15.143863 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:15.144115 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:15.144306 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:15.144306 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:15.144307 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:15.144413 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:15.144655 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:15.144661 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:15.144774 (+ 113us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.144877 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.144991 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":118,"threads_started":1}
I20260925 13:07:15.145763 2762 heartbeater.cc:347] Connected to a master server at 127.0.206.126:46693
I20260925 13:07:15.145860 2762 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:15.146051 2762 heartbeater.cc:511] Master 127.0.206.126:46693 requested a full tablet report, sending...
I20260925 13:07:15.146534 2559 ts_manager.cc:194] Registered new tserver with Master: 4286e8349aa04b22bd638c383898ae9f (127.0.206.65:36595)
I20260925 13:07:15.147310 2559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:35005
I20260925 13:07:15.148494 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 2631
I20260925 13:07:15.148567 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:07:15.148754 825 external_mini_cluster.cc:1428] Setting key 9a157adfbb0c9776769c697fb622a959
I20260925 13:07:15.152583 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.146916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:56769 (local address 127.0.206.126:46693)
0925 13:07:15.146955 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.146957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.147028 (+ 71us) server_negotiation.cc:407] Connection header received
0925 13:07:15.147099 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.147100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.147109 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.147163 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:15.147481 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.148017 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.148657 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.148907 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.148997 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.149907 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:15.149909 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:15.149909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:15.149915 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:15.151576 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:15.151955 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:15.151956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:15.151956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:15.152002 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:15.152191 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:15.152192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:15.152192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:15.152278 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:15.152334 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.152475 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.152530 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 13:07:15.152717 2768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.146847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:46693 (local address 127.0.206.65:56769)
0925 13:07:15.146975 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.146985 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.147044 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.147179 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.147180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.147344 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:15.147398 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.147399 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.148082 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.148083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.148591 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.148593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.148998 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.148999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.149082 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.149519 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:15.149524 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:15.149785 (+ 261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:15.151662 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:15.151663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:15.151663 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:15.151883 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:15.152065 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:15.152066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:15.152066 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:15.152133 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:15.152336 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:15.152337 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:15.152386 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.152461 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.152568 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260925 13:07:15.171607 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:46693
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:34905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:15.252754 2771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:15.252988 2771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:15.253026 2771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:15.254169 2771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:15.254235 2771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:15.254274 2771 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 13:07:15.254302 2771 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 13:07:15.254341 2771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:07:15.255873 2771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:34905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.206.126:46693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:15.256124 2771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:15.256361 2771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:15.261322 2777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:15.261433 2771 server_base.cc:1034] running on GCE node
W20260925 13:07:15.261373 2779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:15.261351 2776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:15.261833 2771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:15.262111 2771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:15.263279 2771 hybrid_clock.cc:648] HybridClock initialized: now 1790341635263266 us; error 45 us; skew 500 ppm
Sep 25 13:07:15 dist-test-slave-chkf krb5kdc[2514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341635, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:15.265599 2771 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:07:15.266088 2771 webserver.cc:460] Webserver started at http://127.0.206.66:45671/ using document root <none> and password file <none>
I20260925 13:07:15.266310 2771 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:15.266378 2771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:15.266525 2771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:15.267575 2771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "01a8bd366b904125a208127c85eddbc2"
format_stamp: "Formatted at 2026-09-25 13:07:15 on dist-test-slave-chkf"
server_key: "142849e77c72c0705b0caf5331816bdc"
server_key_iv: "c727e0cd7d4633cae753105be529f460"
server_key_version: "encryptionkey@0"
I20260925 13:07:15.267951 2771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "01a8bd366b904125a208127c85eddbc2"
format_stamp: "Formatted at 2026-09-25 13:07:15 on dist-test-slave-chkf"
server_key: "142849e77c72c0705b0caf5331816bdc"
server_key_iv: "c727e0cd7d4633cae753105be529f460"
server_key_version: "encryptionkey@0"
I20260925 13:07:15.269256 2771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:07:15.270077 2786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:15.270300 2771 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:15.270403 2771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "01a8bd366b904125a208127c85eddbc2"
format_stamp: "Formatted at 2026-09-25 13:07:15 on dist-test-slave-chkf"
server_key: "142849e77c72c0705b0caf5331816bdc"
server_key_iv: "c727e0cd7d4633cae753105be529f460"
server_key_version: "encryptionkey@0"
I20260925 13:07:15.270488 2771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:15.284288 2771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:15.286077 2771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:15.286234 2771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:15.286764 2771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:15.286821 2771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:15.286862 2771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:15.286921 2771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 13:07:15 dist-test-slave-chkf krb5kdc[2514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341635, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:15.293967 2771 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:43083
I20260925 13:07:15.294238 2901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:43083 every 8 connection(s)
I20260925 13:07:15.294338 2771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:07:15.295737 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 2771
I20260925 13:07:15.295859 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:07:15.296041 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.287201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:58909 (local address 127.0.206.126:46693)
0925 13:07:15.287245 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.287247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.287470 (+ 223us) server_negotiation.cc:407] Connection header received
0925 13:07:15.287971 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.287973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.287985 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.288038 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:15.288685 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.289180 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.290220 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.290458 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.290545 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.293148 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:15.293149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:15.293150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:15.293158 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:15.294883 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:15.295211 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:15.295212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:15.295212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:15.295261 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:15.295432 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:15.295432 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:15.295433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:15.295535 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:15.295584 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.295906 (+ 322us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.295980 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 13:07:15.296059 825 external_mini_cluster.cc:1428] Setting key 3e0263cd5658ea5a712685791bab41f6
I20260925 13:07:15.296247 2795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.287197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:46693 (local address 127.0.206.66:58909)
0925 13:07:15.287356 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.287388 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.287867 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.288108 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.288111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.288329 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:15.288595 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.288600 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.289283 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.289284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.290129 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.290131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.290554 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.290556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.290712 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.291228 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:15.291238 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:15.293055 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:15.294947 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:15.294950 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:15.294951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:15.295144 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:15.295305 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:15.295306 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:15.295306 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:15.295383 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:15.295593 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:15.295597 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:15.295753 (+ 156us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.295873 (+ 120us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.296034 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":102,"threads_started":1}
I20260925 13:07:15.296779 2902 heartbeater.cc:347] Connected to a master server at 127.0.206.126:46693
I20260925 13:07:15.296867 2902 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:15.297037 2902 heartbeater.cc:511] Master 127.0.206.126:46693 requested a full tablet report, sending...
I20260925 13:07:15.297394 2558 ts_manager.cc:194] Registered new tserver with Master: 01a8bd366b904125a208127c85eddbc2 (127.0.206.66:43083)
I20260925 13:07:15.297842 2558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:58909
I20260925 13:07:15.304448 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.298805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:38941 (local address 127.0.206.126:46693)
0925 13:07:15.298865 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.298867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.298875 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:15.298894 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.298895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.298905 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.298954 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:15.299337 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.300046 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.300716 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.300918 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.301009 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.301883 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:15.301884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:15.301885 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:15.301891 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:15.303279 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:15.303635 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:15.303636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:15.303637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:15.303689 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:15.303910 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:15.303911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:15.303912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:15.304061 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:15.304137 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.304293 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.304362 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 13:07:15.304466 2908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.297769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:46693 (local address 127.0.206.66:38941)
0925 13:07:15.297906 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.297916 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.297962 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.299008 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.299009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.299198 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:15.299250 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.299252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.300132 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.300132 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.300617 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.300618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.301010 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.301011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.301085 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.301514 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:15.301519 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:15.301780 (+ 261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:15.303376 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:15.303377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:15.303378 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:15.303550 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:15.303759 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:15.303760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:15.303760 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:15.303815 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:15.304141 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:15.304142 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:15.304185 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.304269 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.304363 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260925 13:07:15.319620 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:46693
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:34905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:15.397928 2911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:15.398124 2911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:15.398149 2911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:15.399327 2911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:15.399367 2911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:15.399387 2911 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 13:07:15.399431 2911 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 13:07:15.399464 2911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:07:15.400882 2911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:34905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.206.126:46693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:15.401116 2911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:15.401335 2911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:15.406100 2916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:15.407569 2911 server_base.cc:1034] running on GCE node
W20260925 13:07:15.407822 2917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:15.407912 2919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:15.408167 2911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:15.408402 2911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:15.409556 2911 hybrid_clock.cc:648] HybridClock initialized: now 1790341635409543 us; error 24 us; skew 500 ppm
Sep 25 13:07:15 dist-test-slave-chkf krb5kdc[2514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341635, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:15.412032 2911 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:07:15.412376 2911 webserver.cc:460] Webserver started at http://127.0.206.67:43119/ using document root <none> and password file <none>
I20260925 13:07:15.412550 2911 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:15.412590 2911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:15.412672 2911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:15.413645 2911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "fcca9f2896b045be991f4117081b10bc"
format_stamp: "Formatted at 2026-09-25 13:07:15 on dist-test-slave-chkf"
server_key: "45a368d2a5f67f7fd5a18ef54748d0fe"
server_key_iv: "7a3fee9b01d5938f4e4e77e58c1b5361"
server_key_version: "encryptionkey@0"
I20260925 13:07:15.413942 2911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "fcca9f2896b045be991f4117081b10bc"
format_stamp: "Formatted at 2026-09-25 13:07:15 on dist-test-slave-chkf"
server_key: "45a368d2a5f67f7fd5a18ef54748d0fe"
server_key_iv: "7a3fee9b01d5938f4e4e77e58c1b5361"
server_key_version: "encryptionkey@0"
I20260925 13:07:15.415304 2911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:07:15.415926 2926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:15.416105 2911 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:07:15.416181 2911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "fcca9f2896b045be991f4117081b10bc"
format_stamp: "Formatted at 2026-09-25 13:07:15 on dist-test-slave-chkf"
server_key: "45a368d2a5f67f7fd5a18ef54748d0fe"
server_key_iv: "7a3fee9b01d5938f4e4e77e58c1b5361"
server_key_version: "encryptionkey@0"
I20260925 13:07:15.416234 2911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:15.446588 2911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:15.448540 2911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:15.448690 2911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:15.449164 2911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:15.449203 2911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:15.449226 2911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:15.449273 2911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:15.455736 2911 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:44053
I20260925 13:07:15.455821 3041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:44053 every 8 connection(s)
I20260925 13:07:15.456168 2911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
Sep 25 13:07:15 dist-test-slave-chkf krb5kdc[2514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341635, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:15.459947 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.449665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:53433 (local address 127.0.206.126:46693)
0925 13:07:15.449714 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.449716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.449911 (+ 195us) server_negotiation.cc:407] Connection header received
0925 13:07:15.450438 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.450440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.450461 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.450510 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:15.451314 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.452113 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.453746 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.453947 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.454003 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.457104 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:15.457106 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:15.457106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:15.457114 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:15.458573 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:15.459014 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:15.459015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:15.459015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:15.459071 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:15.459327 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:15.459328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:15.459329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:15.459442 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:15.459522 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.459783 (+ 261us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.459861 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 13:07:15.460135 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.449644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:46693 (local address 127.0.206.67:53433)
0925 13:07:15.449800 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.449843 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.450344 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.450594 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.450598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.450962 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:15.451230 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.451237 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.452220 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.452221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.453645 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.453648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.454036 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.454038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.454194 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.454782 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:15.454793 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:15.457012 (+ 2219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:15.458688 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:15.458695 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:15.458697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:15.458930 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:15.459147 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:15.459148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:15.459148 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:15.459255 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:15.459514 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:15.459519 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:15.459651 (+ 132us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.459749 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.459899 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":93,"threads_started":1}
I20260925 13:07:15.460582 3042 heartbeater.cc:347] Connected to a master server at 127.0.206.126:46693
I20260925 13:07:15.460678 3042 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:15.460858 3042 heartbeater.cc:511] Master 127.0.206.126:46693 requested a full tablet report, sending...
I20260925 13:07:15.461244 2559 ts_manager.cc:194] Registered new tserver with Master: fcca9f2896b045be991f4117081b10bc (127.0.206.67:44053)
I20260925 13:07:15.461783 2559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:53433
I20260925 13:07:15.463913 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 2911
I20260925 13:07:15.464005 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:07:15.464215 825 external_mini_cluster.cc:1428] Setting key 6f8942f88fdc5555ff8ba4df6d62fad4
I20260925 13:07:15.465271 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:07:15.467176 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.461689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:33387 (local address 127.0.206.126:46693)
0925 13:07:15.461728 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.461729 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.461791 (+ 62us) server_negotiation.cc:407] Connection header received
0925 13:07:15.461861 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.461862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.461871 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.461917 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:15.462253 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.462739 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.463411 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.463613 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.463693 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.464704 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:15.464705 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:15.464705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:15.464712 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:15.466056 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:15.466393 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:15.466394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:15.466395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:15.466442 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:15.466669 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:15.466670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:15.466670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:15.466799 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:15.466875 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.467025 (+ 150us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.467092 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 13:07:15.467185 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.461617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:46693 (local address 127.0.206.67:33387)
0925 13:07:15.461740 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.461750 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.461805 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.461975 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.461976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.462128 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:15.462170 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.462171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.462810 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.462811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.463317 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.463318 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.463694 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.463695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.463792 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.464208 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:15.464213 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:15.464610 (+ 397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:15.466150 (+ 1540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:15.466151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:15.466152 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:15.466329 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:15.466519 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:15.466519 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:15.466520 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:15.466572 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:15.466868 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:15.466869 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:15.466911 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.467003 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.467093 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
Sep 25 13:07:15 dist-test-slave-chkf krb5kdc[2514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 13:07:15 dist-test-slave-chkf krb5kdc[2514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:15.486811 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.480244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43760 (local address 127.0.206.126:46693)
0925 13:07:15.480299 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.480301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.480310 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:15.480369 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.480370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.480379 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.480446 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:15.480977 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.481419 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.482122 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.482295 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.482377 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.484475 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:15.484476 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:15.484476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:15.484482 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:15.485767 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:15.486098 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:15.486098 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:15.486099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:15.486144 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:15.486316 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:15.486316 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:15.486316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:15.486404 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:15.486473 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.486595 (+ 122us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.486648 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 13:07:15.488366 2559 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:07:15.489801 2559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:07:15.496829 3060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.492442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:36595 (local address 127.0.0.1:58568)
0925 13:07:15.492650 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.492659 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.492700 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.493085 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.493086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.493088 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:15.493133 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.493135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.494537 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.494538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.495785 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.495787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.496500 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.496501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.496577 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.496590 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.496666 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.496720 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":75,"threads_started":1}
I20260925 13:07:15.496865 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.492678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58568 (local address 127.0.206.65:36595)
0925 13:07:15.492819 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.492823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.492861 (+ 38us) server_negotiation.cc:407] Connection header received
0925 13:07:15.492902 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.492904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.492949 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.493087 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:15.493217 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.494459 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.495891 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.496412 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.496500 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.496559 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.496682 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.496756 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260925 13:07:15.497094 3062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.492283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:44053 (local address 127.0.0.1:56266)
0925 13:07:15.492550 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.492560 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.492601 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.492916 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.492919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.492922 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:15.492991 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.492993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.494777 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.494778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.495901 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.495903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.496812 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.496813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.496888 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.496891 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.496930 (+ 39us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.496993 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":3,"thread_start_us":64,"threads_started":1}
I20260925 13:07:15.497269 3059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.492207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:43083 (local address 127.0.0.1:37798)
0925 13:07:15.492313 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.492371 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.492429 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.492986 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.492987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.492989 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:15.493029 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.493030 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.494319 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.494320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.495777 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.495780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.496812 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.496813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.496882 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.496885 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.496917 (+ 32us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.496993 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"thread_start_us":46,"threads_started":1}
I20260925 13:07:15.497411 3061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.492329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37798 (local address 127.0.206.66:43083)
0925 13:07:15.492829 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.492833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.492842 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:15.492879 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.492881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.492895 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.493095 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:15.493157 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.494233 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.495896 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.496708 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.497118 (+ 410us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.497187 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.497252 (+ 65us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.497295 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":119,"threads_started":1}
I20260925 13:07:15.497591 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.492460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56266 (local address 127.0.206.67:44053)
0925 13:07:15.492730 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.492735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.492745 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:07:15.492781 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.492783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.492798 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.493068 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:15.493121 (+ 53us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.494221 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.495969 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.496707 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.497371 (+ 664us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.497434 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.497460 (+ 26us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.497503 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":69,"threads_started":1}
I20260925 13:07:15.497901 2696 tablet_service.cc:1467] Processing CreateTablet for tablet ee5ee2415d8d45f5a5275d34fd790f0f (DEFAULT_TABLE table=test-table [id=c6686e6d2ee742df8183bc919d71d751]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:15.497992 2836 tablet_service.cc:1467] Processing CreateTablet for tablet ee5ee2415d8d45f5a5275d34fd790f0f (DEFAULT_TABLE table=test-table [id=c6686e6d2ee742df8183bc919d71d751]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:15.498181 2959 tablet_service.cc:1467] Processing CreateTablet for tablet ee5ee2415d8d45f5a5275d34fd790f0f (DEFAULT_TABLE table=test-table [id=c6686e6d2ee742df8183bc919d71d751]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:15.498255 2696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee5ee2415d8d45f5a5275d34fd790f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:15.498255 2836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee5ee2415d8d45f5a5275d34fd790f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:15.498412 2959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee5ee2415d8d45f5a5275d34fd790f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:15.500773 3065 tablet_bootstrap.cc:492] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc: Bootstrap starting.
I20260925 13:07:15.501678 3065 tablet_bootstrap.cc:654] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:15.501969 3065 log.cc:824] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:15.501986 3066 tablet_bootstrap.cc:492] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2: Bootstrap starting.
I20260925 13:07:15.502413 3067 tablet_bootstrap.cc:492] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f: Bootstrap starting.
I20260925 13:07:15.502568 3066 tablet_bootstrap.cc:654] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:15.502832 3065 tablet_bootstrap.cc:492] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc: No bootstrap required, opened a new log
I20260925 13:07:15.502907 3065 ts_tablet_manager.cc:1397] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:07:15.502926 3066 log.cc:824] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:15.503075 3067 tablet_bootstrap.cc:654] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:15.503427 3067 log.cc:824] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:15.503742 3066 tablet_bootstrap.cc:492] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2: No bootstrap required, opened a new log
I20260925 13:07:15.503815 3066 ts_tablet_manager.cc:1397] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:07:15.504083 3067 tablet_bootstrap.cc:492] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f: No bootstrap required, opened a new log
I20260925 13:07:15.504163 3067 ts_tablet_manager.cc:1397] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 13:07:15.504868 3065 raft_consensus.cc:348] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } }
I20260925 13:07:15.505023 3065 raft_consensus.cc:374] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:15.505057 3065 raft_consensus.cc:729] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcca9f2896b045be991f4117081b10bc, State: Initialized, Role: FOLLOWER
I20260925 13:07:15.505200 3065 consensus_queue.cc:260] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } }
I20260925 13:07:15.505482 3042 heartbeater.cc:503] Master 127.0.206.126:46693 was elected leader, sending a full tablet report...
I20260925 13:07:15.505627 3065 ts_tablet_manager.cc:1428] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260925 13:07:15.505813 3066 raft_consensus.cc:348] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } }
I20260925 13:07:15.505954 3066 raft_consensus.cc:374] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:15.506014 3066 raft_consensus.cc:729] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a8bd366b904125a208127c85eddbc2, State: Initialized, Role: FOLLOWER
I20260925 13:07:15.506125 3066 consensus_queue.cc:260] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } }
I20260925 13:07:15.506163 3067 raft_consensus.cc:348] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } }
I20260925 13:07:15.506366 3067 raft_consensus.cc:374] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:15.506382 3066 ts_tablet_manager.cc:1428] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:07:15.506425 2902 heartbeater.cc:503] Master 127.0.206.126:46693 was elected leader, sending a full tablet report...
I20260925 13:07:15.506445 3067 raft_consensus.cc:729] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4286e8349aa04b22bd638c383898ae9f, State: Initialized, Role: FOLLOWER
I20260925 13:07:15.506577 3067 consensus_queue.cc:260] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } }
I20260925 13:07:15.506860 3067 ts_tablet_manager.cc:1428] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 13:07:15.506906 2762 heartbeater.cc:503] Master 127.0.206.126:46693 was elected leader, sending a full tablet report...
W20260925 13:07:15.544957 2903 tablet.cc:2369] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:07:15.608210 3073 raft_consensus.cc:482] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:15.608321 3073 raft_consensus.cc:504] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } }
I20260925 13:07:15.608745 3073 leader_election.cc:288] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01a8bd366b904125a208127c85eddbc2 (127.0.206.66:43083), fcca9f2896b045be991f4117081b10bc (127.0.206.67:44053)
I20260925 13:07:15.611428 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.608895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:44053 (local address 127.0.206.65:56105)
0925 13:07:15.608944 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.608964 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.609017 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.609179 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.609180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.609182 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:15.609249 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.609251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.609888 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.609889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.610610 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.610612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.611178 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.611179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.611260 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.611263 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.611326 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.611361 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 13:07:15.611428 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.608999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:56105 (local address 127.0.206.67:44053)
0925 13:07:15.609055 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.609057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.609072 (+ 15us) server_negotiation.cc:407] Connection header received
0925 13:07:15.609095 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.609095 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.609102 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.609171 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:15.609337 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.609784 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.610731 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.611078 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.611171 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.611179 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.611333 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.611366 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 13:07:15.611761 3074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.608896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:43083 (local address 127.0.206.65:45081)
0925 13:07:15.609241 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.609249 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.609325 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.609472 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.609473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.609474 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:15.609545 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.609546 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.610175 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.610176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.610891 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.610893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.611476 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.611478 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.611565 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.611569 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.611639 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.611670 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":169,"thread_start_us":118,"threads_started":1}
I20260925 13:07:15.611764 3061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.609001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:45081 (local address 127.0.206.66:43083)
0925 13:07:15.609053 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.609055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.609310 (+ 255us) server_negotiation.cc:407] Connection header received
0925 13:07:15.609390 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.609391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.609399 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.609467 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:15.609611 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.610079 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.610997 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.611368 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.611445 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.611453 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.611661 (+ 208us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.611706 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 13:07:15.611784 2996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee5ee2415d8d45f5a5275d34fd790f0f" candidate_uuid: "4286e8349aa04b22bd638c383898ae9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcca9f2896b045be991f4117081b10bc" is_pre_election: true
I20260925 13:07:15.611949 2996 raft_consensus.cc:2454] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4286e8349aa04b22bd638c383898ae9f in term 0.
I20260925 13:07:15.612135 2856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee5ee2415d8d45f5a5275d34fd790f0f" candidate_uuid: "4286e8349aa04b22bd638c383898ae9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a8bd366b904125a208127c85eddbc2" is_pre_election: true
I20260925 13:07:15.612216 2650 leader_election.cc:302] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4286e8349aa04b22bd638c383898ae9f, fcca9f2896b045be991f4117081b10bc; no voters:
I20260925 13:07:15.612301 2856 raft_consensus.cc:2454] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4286e8349aa04b22bd638c383898ae9f in term 0.
I20260925 13:07:15.612334 3073 raft_consensus.cc:2781] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:07:15.612439 3073 raft_consensus.cc:482] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:07:15.612500 3073 raft_consensus.cc:3037] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:15.613348 3073 raft_consensus.cc:504] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } }
I20260925 13:07:15.613484 3073 leader_election.cc:288] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [CANDIDATE]: Term 1 election: Requested vote from peers 01a8bd366b904125a208127c85eddbc2 (127.0.206.66:43083), fcca9f2896b045be991f4117081b10bc (127.0.206.67:44053)
I20260925 13:07:15.613613 2856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee5ee2415d8d45f5a5275d34fd790f0f" candidate_uuid: "4286e8349aa04b22bd638c383898ae9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a8bd366b904125a208127c85eddbc2"
I20260925 13:07:15.613641 2996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee5ee2415d8d45f5a5275d34fd790f0f" candidate_uuid: "4286e8349aa04b22bd638c383898ae9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcca9f2896b045be991f4117081b10bc"
I20260925 13:07:15.613688 2856 raft_consensus.cc:3037] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:15.613701 2996 raft_consensus.cc:3037] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:15.614490 2856 raft_consensus.cc:2454] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4286e8349aa04b22bd638c383898ae9f in term 1.
I20260925 13:07:15.614490 2996 raft_consensus.cc:2454] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4286e8349aa04b22bd638c383898ae9f in term 1.
I20260925 13:07:15.614652 2647 leader_election.cc:302] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a8bd366b904125a208127c85eddbc2, 4286e8349aa04b22bd638c383898ae9f; no voters:
I20260925 13:07:15.614802 3073 raft_consensus.cc:2781] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:15.614943 3073 raft_consensus.cc:686] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [term 1 LEADER]: Becoming Leader. State: Replica: 4286e8349aa04b22bd638c383898ae9f, State: Running, Role: LEADER
I20260925 13:07:15.615046 3073 consensus_queue.cc:237] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } }
I20260925 13:07:15.615916 2558 catalog_manager.cc:5495] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4286e8349aa04b22bd638c383898ae9f (127.0.206.65). New cstate: current_term: 1 leader_uuid: "4286e8349aa04b22bd638c383898ae9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } health_report { overall_health: HEALTHY } } }
W20260925 13:07:15.644335 2763 tablet.cc:2369] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 13:07:15 dist-test-slave-chkf krb5kdc[2514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341634, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:15.670071 3082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.663298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:46693 (local address 127.0.0.1:43766)
0925 13:07:15.663510 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.663519 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.663602 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.663749 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.663750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.664063 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:15.664112 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.664114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.664752 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.664753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.665254 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.665256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.665594 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.665595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.665644 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.666251 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:15.666255 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:15.667571 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:15.669147 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:15.669152 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:15.669154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:15.669347 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:15.669518 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:15.669518 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:15.669519 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:15.669571 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:15.669768 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:15.669773 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:15.669816 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.669875 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.669984 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":14,"thread_start_us":124,"threads_started":1}
I20260925 13:07:15.670117 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.663362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43766 (local address 127.0.206.126:46693)
0925 13:07:15.663446 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.663448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.663583 (+ 135us) server_negotiation.cc:407] Connection header received
0925 13:07:15.663661 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.663661 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.663672 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.663744 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:15.664205 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.664672 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.665340 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.665513 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.665595 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.667653 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:15.667654 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:15.667655 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:15.667661 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:15.669070 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:15.669413 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:15.669414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:15.669414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:15.669461 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:15.669622 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:15.669623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:15.669623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:15.669718 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:15.669768 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.669891 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.670023 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":29}
I20260925 13:07:15.672027 2558 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:43766:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 13:07:15.672219 2558 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:07:15.673789 2836 tablet_service.cc:1467] Processing CreateTablet for tablet c2ed089afd064215a4b88d46cc022dc0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=536ee0ad8589414f9ca70a40226a447a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 13:07:15.673791 2696 tablet_service.cc:1467] Processing CreateTablet for tablet c2ed089afd064215a4b88d46cc022dc0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=536ee0ad8589414f9ca70a40226a447a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 13:07:15.673952 2836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2ed089afd064215a4b88d46cc022dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:15.673952 2696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2ed089afd064215a4b88d46cc022dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:15.675442 3067 tablet_bootstrap.cc:492] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f: Bootstrap starting.
I20260925 13:07:15.675442 3066 tablet_bootstrap.cc:492] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2: Bootstrap starting.
I20260925 13:07:15.675557 2959 tablet_service.cc:1467] Processing CreateTablet for tablet c2ed089afd064215a4b88d46cc022dc0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=536ee0ad8589414f9ca70a40226a447a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 13:07:15.675720 2959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2ed089afd064215a4b88d46cc022dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:15.675935 3067 tablet_bootstrap.cc:654] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:15.676120 3066 tablet_bootstrap.cc:654] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:15.676566 3067 tablet_bootstrap.cc:492] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f: No bootstrap required, opened a new log
I20260925 13:07:15.676632 3067 ts_tablet_manager.cc:1397] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 13:07:15.676802 3066 tablet_bootstrap.cc:492] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2: No bootstrap required, opened a new log
I20260925 13:07:15.676779 3067 raft_consensus.cc:348] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:15.676838 3066 ts_tablet_manager.cc:1397] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:15.676858 3067 raft_consensus.cc:374] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:15.676884 3067 raft_consensus.cc:729] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4286e8349aa04b22bd638c383898ae9f, State: Initialized, Role: FOLLOWER
I20260925 13:07:15.676939 3067 consensus_queue.cc:260] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:15.677021 3067 ts_tablet_manager.cc:1428] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:15.676992 3066 raft_consensus.cc:348] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:15.677079 3066 raft_consensus.cc:374] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:15.677106 3066 raft_consensus.cc:729] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a8bd366b904125a208127c85eddbc2, State: Initialized, Role: FOLLOWER
I20260925 13:07:15.677058 3073 tablet_replica.cc:439] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } }
I20260925 13:07:15.677141 3073 tablet_replica.cc:442] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:15.677157 3065 tablet_bootstrap.cc:492] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc: Bootstrap starting.
I20260925 13:07:15.677152 3066 consensus_queue.cc:260] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:15.677249 3066 ts_tablet_manager.cc:1428] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:15.677681 3065 tablet_bootstrap.cc:654] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:15.678102 3072 tablet_replica.cc:439] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } }
I20260925 13:07:15.678181 3072 tablet_replica.cc:442] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:15.678326 3065 tablet_bootstrap.cc:492] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc: No bootstrap required, opened a new log
I20260925 13:07:15.678370 3065 ts_tablet_manager.cc:1397] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 13:07:15.678507 3065 raft_consensus.cc:348] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:15.678584 3065 raft_consensus.cc:374] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:15.678608 3065 raft_consensus.cc:729] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcca9f2896b045be991f4117081b10bc, State: Initialized, Role: FOLLOWER
I20260925 13:07:15.678655 3065 consensus_queue.cc:260] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:15.678782 3065 ts_tablet_manager.cc:1428] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 13:07:15.678866 3071 tablet_replica.cc:439] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } }
I20260925 13:07:15.678932 3071 tablet_replica.cc:442] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc: This TxnStatusTablet replica's current role is: FOLLOWER
W20260925 13:07:15.706732 3043 tablet.cc:2369] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:07:15.707007 3072 raft_consensus.cc:482] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:15.707082 3072 raft_consensus.cc:504] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:15.707396 3072 leader_election.cc:288] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4286e8349aa04b22bd638c383898ae9f (127.0.206.65:36595), fcca9f2896b045be991f4117081b10bc (127.0.206.67:44053)
I20260925 13:07:15.711194 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.707666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:43359 (local address 127.0.206.65:36595)
0925 13:07:15.707710 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.707712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.707759 (+ 47us) server_negotiation.cc:407] Connection header received
0925 13:07:15.707801 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.707802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.707809 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.707849 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:15.708028 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.708784 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.710263 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.710837 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.710908 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.710916 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.711092 (+ 176us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.711125 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 13:07:15.711194 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.707674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:33309 (local address 127.0.206.67:44053)
0925 13:07:15.707718 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.707720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.707894 (+ 174us) server_negotiation.cc:407] Connection header received
0925 13:07:15.707925 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.707926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.707934 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.708005 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:15.708131 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.708811 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.710269 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.710833 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.710908 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.710916 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.711092 (+ 176us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.711126 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 13:07:15.711228 2795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.707573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:36595 (local address 127.0.206.66:43359)
0925 13:07:15.707648 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.707660 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.707701 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.707877 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.707877 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.707879 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:15.707955 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.707957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.708884 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.708884 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.710188 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.710191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.710917 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.710918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.711010 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.711014 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.711076 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.711130 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":76}
I20260925 13:07:15.711237 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.707604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:44053 (local address 127.0.206.66:33309)
0925 13:07:15.707794 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.707808 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.707878 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.708004 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.708005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.708007 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:15.708072 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.708073 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.708884 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.708884 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.710188 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.710191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.710936 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.710937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.711009 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.711015 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.711076 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.711130 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":61,"thread_start_us":96,"threads_started":1}
I20260925 13:07:15.711584 2996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2ed089afd064215a4b88d46cc022dc0" candidate_uuid: "01a8bd366b904125a208127c85eddbc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcca9f2896b045be991f4117081b10bc" is_pre_election: true
I20260925 13:07:15.711690 2996 raft_consensus.cc:2454] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01a8bd366b904125a208127c85eddbc2 in term 0.
I20260925 13:07:15.711740 2716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2ed089afd064215a4b88d46cc022dc0" candidate_uuid: "01a8bd366b904125a208127c85eddbc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4286e8349aa04b22bd638c383898ae9f" is_pre_election: true
I20260925 13:07:15.711858 2716 raft_consensus.cc:2454] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01a8bd366b904125a208127c85eddbc2 in term 0.
I20260925 13:07:15.712044 2790 leader_election.cc:302] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a8bd366b904125a208127c85eddbc2, fcca9f2896b045be991f4117081b10bc; no voters:
I20260925 13:07:15.712198 3072 raft_consensus.cc:2781] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:07:15.712250 3072 raft_consensus.cc:482] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:07:15.712296 3072 raft_consensus.cc:3037] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:15.712970 3072 raft_consensus.cc:504] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:15.713106 3072 leader_election.cc:288] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [CANDIDATE]: Term 1 election: Requested vote from peers 4286e8349aa04b22bd638c383898ae9f (127.0.206.65:36595), fcca9f2896b045be991f4117081b10bc (127.0.206.67:44053)
I20260925 13:07:15.713256 2716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2ed089afd064215a4b88d46cc022dc0" candidate_uuid: "01a8bd366b904125a208127c85eddbc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4286e8349aa04b22bd638c383898ae9f"
I20260925 13:07:15.713261 2996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2ed089afd064215a4b88d46cc022dc0" candidate_uuid: "01a8bd366b904125a208127c85eddbc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcca9f2896b045be991f4117081b10bc"
I20260925 13:07:15.713335 2716 raft_consensus.cc:3037] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:15.713356 2996 raft_consensus.cc:3037] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:15.713943 2996 raft_consensus.cc:2454] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a8bd366b904125a208127c85eddbc2 in term 1.
I20260925 13:07:15.714008 2716 raft_consensus.cc:2454] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a8bd366b904125a208127c85eddbc2 in term 1.
I20260925 13:07:15.714151 2790 leader_election.cc:302] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a8bd366b904125a208127c85eddbc2, fcca9f2896b045be991f4117081b10bc; no voters:
I20260925 13:07:15.714228 3072 raft_consensus.cc:2781] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:15.714399 3072 raft_consensus.cc:686] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [term 1 LEADER]: Becoming Leader. State: Replica: 01a8bd366b904125a208127c85eddbc2, State: Running, Role: LEADER
I20260925 13:07:15.714493 3072 consensus_queue.cc:237] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:15.714950 3087 tablet_replica.cc:439] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2: TxnStatusTablet state changed. Reason: New leader 01a8bd366b904125a208127c85eddbc2. Latest consensus state: current_term: 1 leader_uuid: "01a8bd366b904125a208127c85eddbc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } }
I20260925 13:07:15.715040 3087 tablet_replica.cc:442] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2: This TxnStatusTablet replica's current role is: LEADER
I20260925 13:07:15.715226 3089 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 13:07:15.715337 3089 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 13:07:15.715283 2559 catalog_manager.cc:5495] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01a8bd366b904125a208127c85eddbc2 (127.0.206.66). New cstate: current_term: 1 leader_uuid: "01a8bd366b904125a208127c85eddbc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } health_report { overall_health: UNKNOWN } } }
I20260925 13:07:15.724416 3082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.721677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:43083 (local address 127.0.0.1:37800)
0925 13:07:15.721744 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.721757 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.721809 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.722014 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.722015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.722016 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 13:07:15.722054 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.722056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.722953 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.722953 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.723503 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.723504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.723961 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.723961 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.724006 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.724019 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 13:07:15.724287 (+ 268us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 13:07:15.724293 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.724329 (+ 36us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.724363 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 13:07:15.724426 3061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.721812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37800 (local address 127.0.206.66:43083)
0925 13:07:15.721875 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.721878 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.721886 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:15.721914 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.721915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.721932 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.722014 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:15.722112 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.722880 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.723582 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.723885 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.723957 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.724101 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:15.724215 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:15.724296 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.724335 (+ 39us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.724356 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 13:07:15.726188 2716 raft_consensus.cc:1261] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f [term 1 FOLLOWER]: Refusing update from remote peer 01a8bd366b904125a208127c85eddbc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:15.726222 2996 raft_consensus.cc:1261] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc [term 1 FOLLOWER]: Refusing update from remote peer 01a8bd366b904125a208127c85eddbc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:15.726455 3072 consensus_queue.cc:1035] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:15.726552 3087 consensus_queue.cc:1035] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:15.727589 3073 tablet_replica.cc:439] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f: TxnStatusTablet state changed. Reason: New leader 01a8bd366b904125a208127c85eddbc2. Latest consensus state: current_term: 1 leader_uuid: "01a8bd366b904125a208127c85eddbc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } }
I20260925 13:07:15.727679 3073 tablet_replica.cc:442] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:15.727700 3071 tablet_replica.cc:439] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc: TxnStatusTablet state changed. Reason: New leader 01a8bd366b904125a208127c85eddbc2. Latest consensus state: current_term: 1 leader_uuid: "01a8bd366b904125a208127c85eddbc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } }
I20260925 13:07:15.727837 3071 tablet_replica.cc:442] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:15.731594 3072 tablet_replica.cc:439] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "01a8bd366b904125a208127c85eddbc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } }
I20260925 13:07:15.731673 3072 tablet_replica.cc:442] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2: This TxnStatusTablet replica's current role is: LEADER
I20260925 13:07:15.731763 3072 tablet_replica.cc:439] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "01a8bd366b904125a208127c85eddbc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } }
I20260925 13:07:15.731817 3072 tablet_replica.cc:442] T c2ed089afd064215a4b88d46cc022dc0 P 01a8bd366b904125a208127c85eddbc2: This TxnStatusTablet replica's current role is: LEADER
I20260925 13:07:15.732136 3073 tablet_replica.cc:439] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "01a8bd366b904125a208127c85eddbc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } }
I20260925 13:07:15.732214 3073 tablet_replica.cc:442] T c2ed089afd064215a4b88d46cc022dc0 P 4286e8349aa04b22bd638c383898ae9f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:15.732908 3071 tablet_replica.cc:439] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "01a8bd366b904125a208127c85eddbc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } } }
I20260925 13:07:15.732991 3071 tablet_replica.cc:442] T c2ed089afd064215a4b88d46cc022dc0 P fcca9f2896b045be991f4117081b10bc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:15.733510 3093 mvcc.cc:205] Tried to move back new op lower bound from 7333239339931901952 to 7333239339887038464. Current Snapshot: MvccSnapshot[applied={T|T < 7333239339931901952 or (T in {7333239339931901952})}]
I20260925 13:07:15.736657 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.734347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58584 (local address 127.0.206.65:36595)
0925 13:07:15.734384 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.734386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.734393 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:07:15.734452 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.734453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.734469 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.734534 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:15.734633 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.735128 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.735912 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.736112 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.736182 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.736348 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:15.736442 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:15.736519 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.736569 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.736589 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 13:07:15.741513 3061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.738919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:47499 (local address 127.0.206.66:43083)
0925 13:07:15.738955 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.738957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.739090 (+ 133us) server_negotiation.cc:407] Connection header received
0925 13:07:15.739148 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.739149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.739159 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.739213 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:15.739329 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.740045 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.740777 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.741072 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.741145 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.741302 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:15.741332 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:15.741386 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.741430 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.741447 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 13:07:15.741520 3103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.738863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:43083 (local address 127.0.206.65:47499)
0925 13:07:15.739040 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.739049 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.739095 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.739214 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.739215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.739216 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 13:07:15.739266 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.739267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.740136 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.740136 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.740701 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.740703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.741154 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.741155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.741205 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.741216 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 13:07:15.741391 (+ 175us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 13:07:15.741393 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.741427 (+ 34us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.741458 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":68,"threads_started":1}
I20260925 13:07:15.743551 2996 raft_consensus.cc:1261] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc [term 1 FOLLOWER]: Refusing update from remote peer 4286e8349aa04b22bd638c383898ae9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:15.743664 2856 raft_consensus.cc:1261] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2 [term 1 FOLLOWER]: Refusing update from remote peer 4286e8349aa04b22bd638c383898ae9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:15.743794 3073 consensus_queue.cc:1035] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:15.743902 3075 consensus_queue.cc:1035] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:15.745541 3094 mvcc.cc:205] Tried to move back new op lower bound from 7333239340004134912 to 7333239339479990272. Current Snapshot: MvccSnapshot[applied={T|T < 7333239340004134912}]
I20260925 13:07:15.751613 3092 mvcc.cc:205] Tried to move back new op lower bound from 7333239340004134912 to 7333239339479990272. Current Snapshot: MvccSnapshot[applied={T|T < 7333239340004134912}]
I20260925 13:07:15.752431 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.749240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:50221 (local address 127.0.206.65:36595)
0925 13:07:15.749286 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.749287 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.749343 (+ 56us) server_negotiation.cc:407] Connection header received
0925 13:07:15.749411 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.749411 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.749419 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.749455 (+ 36us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:15.749574 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.749983 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.751674 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.751980 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.752048 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.752221 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:15.752252 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:15.752297 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.752354 (+ 57us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.752371 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:07:15.752454 2908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.749179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:36595 (local address 127.0.206.66:50221)
0925 13:07:15.749296 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.749307 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.749357 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.749468 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.749469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.749470 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 13:07:15.749523 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.749524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.750049 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.750050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.751014 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.751017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.752068 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.752069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.752140 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.752154 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 13:07:15.752305 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 13:07:15.752307 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.752352 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.752389 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260925 13:07:15.761062 2559 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:15.761163 2559 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:15.761818 2559 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9: Sending DeleteTablet for 3 replicas of tablet ee5ee2415d8d45f5a5275d34fd790f0f
I20260925 13:07:15.762071 2836 tablet_service.cc:1514] Processing DeleteTablet for tablet ee5ee2415d8d45f5a5275d34fd790f0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:15 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:37798
I20260925 13:07:15.762135 2959 tablet_service.cc:1514] Processing DeleteTablet for tablet ee5ee2415d8d45f5a5275d34fd790f0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:15 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:56266
I20260925 13:07:15.762459 2696 tablet_service.cc:1514] Processing DeleteTablet for tablet ee5ee2415d8d45f5a5275d34fd790f0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:15 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:58568
I20260925 13:07:15.762416 2559 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:07:15.762553 2559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:07:15.762859 3113 tablet_replica.cc:330] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f: stopping tablet replica
I20260925 13:07:15.763010 3113 raft_consensus.cc:2229] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:07:15.763160 3113 raft_consensus.cc:2258] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:15.763772 2959 tablet_service.cc:1467] Processing CreateTablet for tablet 526452c8010c40ab921043fc0fc5704f (DEFAULT_TABLE table=test-table [id=ce3b207f4c9743619cad0f7fd54d4c4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:15.763788 2836 tablet_service.cc:1467] Processing CreateTablet for tablet 526452c8010c40ab921043fc0fc5704f (DEFAULT_TABLE table=test-table [id=ce3b207f4c9743619cad0f7fd54d4c4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:15.763890 2959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 526452c8010c40ab921043fc0fc5704f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:15.763919 2836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 526452c8010c40ab921043fc0fc5704f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:15.764780 3065 tablet_bootstrap.cc:492] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc: Bootstrap starting.
I20260925 13:07:15.764915 2696 tablet_service.cc:1467] Processing CreateTablet for tablet 526452c8010c40ab921043fc0fc5704f (DEFAULT_TABLE table=test-table [id=ce3b207f4c9743619cad0f7fd54d4c4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:15.765026 2696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 526452c8010c40ab921043fc0fc5704f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:15.765136 3066 tablet_bootstrap.cc:492] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2: Bootstrap starting.
I20260925 13:07:15.765159 3065 tablet_bootstrap.cc:654] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:15.765523 3066 tablet_bootstrap.cc:654] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:15.766160 3066 tablet_bootstrap.cc:492] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2: No bootstrap required, opened a new log
I20260925 13:07:15.766206 3066 ts_tablet_manager.cc:1397] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:15.766222 3067 tablet_bootstrap.cc:492] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f: Bootstrap starting.
I20260925 13:07:15.766336 3066 raft_consensus.cc:348] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:15.766393 3066 raft_consensus.cc:374] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:15.766413 3066 raft_consensus.cc:729] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a8bd366b904125a208127c85eddbc2, State: Initialized, Role: FOLLOWER
I20260925 13:07:15.766451 3066 consensus_queue.cc:260] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:15.766544 3066 ts_tablet_manager.cc:1428] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:15.766618 3067 tablet_bootstrap.cc:654] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:15.767083 3113 ts_tablet_manager.cc:1905] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:15.767259 3067 tablet_bootstrap.cc:492] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f: No bootstrap required, opened a new log
I20260925 13:07:15.767309 3067 ts_tablet_manager.cc:1397] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 13:07:15.767441 3067 raft_consensus.cc:348] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:15.767520 3067 raft_consensus.cc:374] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:15.767547 3067 raft_consensus.cc:729] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4286e8349aa04b22bd638c383898ae9f, State: Initialized, Role: FOLLOWER
I20260925 13:07:15.767587 3067 consensus_queue.cc:260] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:15.767666 3067 ts_tablet_manager.cc:1428] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:15.769145 3113 ts_tablet_manager.cc:1918] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 13:07:15.769186 3065 tablet_bootstrap.cc:492] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc: No bootstrap required, opened a new log
I20260925 13:07:15.769210 3113 log.cc:1196] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/wal/wals/ee5ee2415d8d45f5a5275d34fd790f0f
I20260925 13:07:15.769223 3065 ts_tablet_manager.cc:1397] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260925 13:07:15.769357 3065 raft_consensus.cc:348] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:15.769426 3065 raft_consensus.cc:374] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:15.769449 3065 raft_consensus.cc:729] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcca9f2896b045be991f4117081b10bc, State: Initialized, Role: FOLLOWER
I20260925 13:07:15.769465 3113 ts_tablet_manager.cc:1939] T ee5ee2415d8d45f5a5275d34fd790f0f P 4286e8349aa04b22bd638c383898ae9f: Deleting consensus metadata
I20260925 13:07:15.769508 3065 consensus_queue.cc:260] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:15.769604 3065 ts_tablet_manager.cc:1428] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 13:07:15.770238 2543 catalog_manager.cc:4841] TS 4286e8349aa04b22bd638c383898ae9f (127.0.206.65:36595): tablet ee5ee2415d8d45f5a5275d34fd790f0f (table test-table [id=c6686e6d2ee742df8183bc919d71d751]) successfully deleted
I20260925 13:07:15.770781 3111 tablet_replica.cc:330] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2: stopping tablet replica
I20260925 13:07:15.770841 3112 tablet_replica.cc:330] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc: stopping tablet replica
I20260925 13:07:15.770890 3111 raft_consensus.cc:2229] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:15.770951 3111 raft_consensus.cc:2258] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:15.770969 3112 raft_consensus.cc:2229] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:15.771054 3112 raft_consensus.cc:2258] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:15.771400 3112 ts_tablet_manager.cc:1905] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:15.772608 3112 ts_tablet_manager.cc:1918] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 13:07:15.772702 3112 log.cc:1196] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/ee5ee2415d8d45f5a5275d34fd790f0f
I20260925 13:07:15.772948 3112 ts_tablet_manager.cc:1939] T ee5ee2415d8d45f5a5275d34fd790f0f P fcca9f2896b045be991f4117081b10bc: Deleting consensus metadata
I20260925 13:07:15.773299 2546 catalog_manager.cc:4841] TS fcca9f2896b045be991f4117081b10bc (127.0.206.67:44053): tablet ee5ee2415d8d45f5a5275d34fd790f0f (table test-table [id=c6686e6d2ee742df8183bc919d71d751]) successfully deleted
I20260925 13:07:15.775009 3111 ts_tablet_manager.cc:1905] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:15.776312 3111 ts_tablet_manager.cc:1918] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 13:07:15.776391 3111 log.cc:1196] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/ee5ee2415d8d45f5a5275d34fd790f0f
I20260925 13:07:15.776835 3111 ts_tablet_manager.cc:1939] T ee5ee2415d8d45f5a5275d34fd790f0f P 01a8bd366b904125a208127c85eddbc2: Deleting consensus metadata
I20260925 13:07:15.777189 2543 catalog_manager.cc:4841] TS 01a8bd366b904125a208127c85eddbc2 (127.0.206.66:43083): tablet ee5ee2415d8d45f5a5275d34fd790f0f (table test-table [id=c6686e6d2ee742df8183bc919d71d751]) successfully deleted
I20260925 13:07:15.972365 3071 raft_consensus.cc:482] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:15.972451 3071 raft_consensus.cc:504] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:15.972695 3071 leader_election.cc:288] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4286e8349aa04b22bd638c383898ae9f (127.0.206.65:36595), 01a8bd366b904125a208127c85eddbc2 (127.0.206.66:43083)
I20260925 13:07:15.975790 3061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.973271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:45823 (local address 127.0.206.66:43083)
0925 13:07:15.973332 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.973334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.973340 (+ 6us) server_negotiation.cc:407] Connection header received
0925 13:07:15.973384 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.973385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.973395 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.973450 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:15.973607 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.974128 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.975114 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.975458 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.975537 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.975546 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.975680 (+ 134us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.975713 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 13:07:15.975801 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.973018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:48759 (local address 127.0.206.65:36595)
0925 13:07:15.973077 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.973079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.973208 (+ 129us) server_negotiation.cc:407] Connection header received
0925 13:07:15.973306 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.973307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.973323 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.973383 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:15.973575 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.974109 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.975144 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.975471 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.975546 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.975554 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.975694 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.975726 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 13:07:15.975822 3117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.972975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:36595 (local address 127.0.206.67:48759)
0925 13:07:15.973122 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.973132 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.973225 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.973405 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.973406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.973409 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:15.973486 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.973488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.974187 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.974187 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.975050 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.975052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.975545 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.975545 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.975610 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.975616 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.975687 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.975727 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":26,"thread_start_us":59,"threads_started":1}
I20260925 13:07:15.975839 3118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.973154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:43083 (local address 127.0.206.67:45823)
0925 13:07:15.973232 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.973241 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.973316 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.973464 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.973465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.973467 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:15.973547 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.973548 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.974208 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.974209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.975020 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.975022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.975535 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.975536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:15.975601 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.975604 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:07:15.975671 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 13:07:15.975716 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"thread_start_us":34,"threads_started":1}
I20260925 13:07:15.976065 2716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "526452c8010c40ab921043fc0fc5704f" candidate_uuid: "fcca9f2896b045be991f4117081b10bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4286e8349aa04b22bd638c383898ae9f" is_pre_election: true
I20260925 13:07:15.976158 2716 raft_consensus.cc:2454] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcca9f2896b045be991f4117081b10bc in term 0.
I20260925 13:07:15.976148 2856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "526452c8010c40ab921043fc0fc5704f" candidate_uuid: "fcca9f2896b045be991f4117081b10bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a8bd366b904125a208127c85eddbc2" is_pre_election: true
I20260925 13:07:15.976228 2856 raft_consensus.cc:2454] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcca9f2896b045be991f4117081b10bc in term 0.
I20260925 13:07:15.976352 2927 leader_election.cc:302] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4286e8349aa04b22bd638c383898ae9f, fcca9f2896b045be991f4117081b10bc; no voters:
I20260925 13:07:15.976485 3071 raft_consensus.cc:2781] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:07:15.976521 3071 raft_consensus.cc:482] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:07:15.976534 3071 raft_consensus.cc:3037] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:15.977142 3071 raft_consensus.cc:504] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:15.977237 3071 leader_election.cc:288] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [CANDIDATE]: Term 1 election: Requested vote from peers 4286e8349aa04b22bd638c383898ae9f (127.0.206.65:36595), 01a8bd366b904125a208127c85eddbc2 (127.0.206.66:43083)
I20260925 13:07:15.977397 2716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "526452c8010c40ab921043fc0fc5704f" candidate_uuid: "fcca9f2896b045be991f4117081b10bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4286e8349aa04b22bd638c383898ae9f"
I20260925 13:07:15.977494 2716 raft_consensus.cc:3037] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:15.977465 2856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "526452c8010c40ab921043fc0fc5704f" candidate_uuid: "fcca9f2896b045be991f4117081b10bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a8bd366b904125a208127c85eddbc2"
I20260925 13:07:15.977527 2856 raft_consensus.cc:3037] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:15.978281 2856 raft_consensus.cc:2454] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcca9f2896b045be991f4117081b10bc in term 1.
I20260925 13:07:15.978300 2716 raft_consensus.cc:2454] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcca9f2896b045be991f4117081b10bc in term 1.
I20260925 13:07:15.978446 2927 leader_election.cc:302] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a8bd366b904125a208127c85eddbc2, fcca9f2896b045be991f4117081b10bc; no voters:
I20260925 13:07:15.978564 3071 raft_consensus.cc:2781] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:15.978737 3071 raft_consensus.cc:686] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [term 1 LEADER]: Becoming Leader. State: Replica: fcca9f2896b045be991f4117081b10bc, State: Running, Role: LEADER
I20260925 13:07:15.978809 3071 consensus_queue.cc:237] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:15.979415 2559 catalog_manager.cc:5495] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc reported cstate change: term changed from 0 to 1, leader changed from <none> to fcca9f2896b045be991f4117081b10bc (127.0.206.67). New cstate: current_term: 1 leader_uuid: "fcca9f2896b045be991f4117081b10bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } health_report { overall_health: HEALTHY } } }
I20260925 13:07:15.996946 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.994741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56280 (local address 127.0.206.67:44053)
0925 13:07:15.994793 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.994795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.994806 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:07:15.994877 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.994878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.994896 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.994962 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:15.995068 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.995481 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.996188 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:15.996434 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:15.996501 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:15.996643 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:15.996742 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:15.996821 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:15.996861 (+ 40us) server_negotiation.cc:299] Negotiation successful
0925 13:07:15.996879 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 13:07:16.001224 3061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.999315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:52321 (local address 127.0.206.66:43083)
0925 13:07:15.999350 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:15.999352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:15.999416 (+ 64us) server_negotiation.cc:407] Connection header received
0925 13:07:15.999463 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:15.999464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:15.999471 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:15.999512 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:15.999600 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:16.000032 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:16.000692 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:16.000862 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:16.000909 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:16.001046 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:16.001068 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:16.001105 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:16.001152 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 13:07:16.001180 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:07:16.001243 3123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:15.999240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:43083 (local address 127.0.206.67:52321)
0925 13:07:15.999358 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:15.999368 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:15.999427 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:15.999521 (+ 94us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:15.999522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:15.999523 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 13:07:15.999563 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:15.999563 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:16.000089 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:16.000090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:16.000626 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:16.000627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:16.000921 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:16.000921 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:16.000969 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:16.000980 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 13:07:16.001116 (+ 136us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 13:07:16.001118 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:07:16.001151 (+ 33us) client_negotiation.cc:240] Negotiation successful
0925 13:07:16.001191 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":49,"threads_started":1}
I20260925 13:07:16.003044 2856 raft_consensus.cc:1261] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2 [term 1 FOLLOWER]: Refusing update from remote peer fcca9f2896b045be991f4117081b10bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:16.003069 2716 raft_consensus.cc:1261] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f [term 1 FOLLOWER]: Refusing update from remote peer fcca9f2896b045be991f4117081b10bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:16.003257 3071 consensus_queue.cc:1035] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:16.003409 3119 consensus_queue.cc:1035] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:16.010274 3063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:16.008225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:42003 (local address 127.0.206.67:44053)
0925 13:07:16.008273 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:16.008275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:16.008284 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:16.008305 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:16.008305 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:16.008313 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:16.008359 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:16.008467 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:16.008908 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:16.009571 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:16.009869 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:16.009948 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:16.010073 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:16.010101 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:16.010146 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:16.010194 (+ 48us) server_negotiation.cc:299] Negotiation successful
0925 13:07:16.010208 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 13:07:16.010300 2908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:16.008147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:44053 (local address 127.0.206.66:42003)
0925 13:07:16.008197 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:16.008208 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:16.008248 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:16.008359 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:16.008359 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:16.008360 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 13:07:16.008397 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:16.008399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:16.008984 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:16.008985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:16.009484 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:16.009486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:16.009950 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:16.009950 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:16.010004 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:16.010008 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 13:07:16.010146 (+ 138us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 13:07:16.010147 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 13:07:16.010192 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 13:07:16.010232 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 13:07:16.010572 3094 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 13:07:17.528409 2559 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:17.528499 2559 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:17.529285 2559 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9: Sending DeleteTablet for 3 replicas of tablet 526452c8010c40ab921043fc0fc5704f
I20260925 13:07:17.529556 2836 tablet_service.cc:1514] Processing DeleteTablet for tablet 526452c8010c40ab921043fc0fc5704f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:17 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:37798
I20260925 13:07:17.529578 2959 tablet_service.cc:1514] Processing DeleteTablet for tablet 526452c8010c40ab921043fc0fc5704f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:17 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:56266
I20260925 13:07:17.529578 2696 tablet_service.cc:1514] Processing DeleteTablet for tablet 526452c8010c40ab921043fc0fc5704f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:17 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:58568
I20260925 13:07:17.530025 3152 tablet_replica.cc:330] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f: stopping tablet replica
I20260925 13:07:17.530078 3152 raft_consensus.cc:2229] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:17.530103 3152 raft_consensus.cc:2258] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:17.530339 3152 ts_tablet_manager.cc:1905] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:17.530818 3153 tablet_replica.cc:330] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc: stopping tablet replica
I20260925 13:07:17.530951 3151 tablet_replica.cc:330] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2: stopping tablet replica
I20260925 13:07:17.531050 3151 raft_consensus.cc:2229] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:17.531139 3153 raft_consensus.cc:2229] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:07:17.531226 3151 raft_consensus.cc:2258] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:17.531368 3153 raft_consensus.cc:2258] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:17.531630 3151 ts_tablet_manager.cc:1905] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:17.532116 3152 ts_tablet_manager.cc:1918] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 13:07:17.532181 3152 log.cc:1196] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/wal/wals/526452c8010c40ab921043fc0fc5704f
I20260925 13:07:17.532399 3152 ts_tablet_manager.cc:1939] T 526452c8010c40ab921043fc0fc5704f P 4286e8349aa04b22bd638c383898ae9f: Deleting consensus metadata
I20260925 13:07:17.532718 2543 catalog_manager.cc:4841] TS 4286e8349aa04b22bd638c383898ae9f (127.0.206.65:36595): tablet 526452c8010c40ab921043fc0fc5704f (table test-table [id=ce3b207f4c9743619cad0f7fd54d4c4b]) successfully deleted
I20260925 13:07:17.532742 3153 ts_tablet_manager.cc:1905] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:17.533246 3151 ts_tablet_manager.cc:1918] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 13:07:17.533308 3151 log.cc:1196] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/526452c8010c40ab921043fc0fc5704f
I20260925 13:07:17.533525 3151 ts_tablet_manager.cc:1939] T 526452c8010c40ab921043fc0fc5704f P 01a8bd366b904125a208127c85eddbc2: Deleting consensus metadata
I20260925 13:07:17.533876 2543 catalog_manager.cc:4841] TS 01a8bd366b904125a208127c85eddbc2 (127.0.206.66:43083): tablet 526452c8010c40ab921043fc0fc5704f (table test-table [id=ce3b207f4c9743619cad0f7fd54d4c4b]) successfully deleted
I20260925 13:07:17.534440 3153 ts_tablet_manager.cc:1918] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 13:07:17.534503 3153 log.cc:1196] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/526452c8010c40ab921043fc0fc5704f
I20260925 13:07:17.534770 3153 ts_tablet_manager.cc:1939] T 526452c8010c40ab921043fc0fc5704f P fcca9f2896b045be991f4117081b10bc: Deleting consensus metadata
I20260925 13:07:17.535061 2546 catalog_manager.cc:4841] TS fcca9f2896b045be991f4117081b10bc (127.0.206.67:44053): tablet 526452c8010c40ab921043fc0fc5704f (table test-table [id=ce3b207f4c9743619cad0f7fd54d4c4b]) successfully deleted
Sep 25 13:07:17 dist-test-slave-chkf krb5kdc[2514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341637, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 13:07:17 dist-test-slave-chkf krb5kdc[2514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341637, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:17.561058 3161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:17.548557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43782 (local address 127.0.206.126:46693)
0925 13:07:17.548686 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:17.548688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:17.548731 (+ 43us) server_negotiation.cc:407] Connection header received
0925 13:07:17.548789 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:17.548789 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:17.548799 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:17.548861 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:17.549361 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:17.549841 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:17.550531 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:17.550755 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:17.550806 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:17.558653 (+ 7847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:17.558654 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:17.558654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:17.558660 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:17.559962 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:17.560255 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:17.560256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:17.560256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:17.560341 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:17.560551 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:17.560552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:17.560552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:17.560663 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:17.560757 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:17.560888 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 13:07:17.560950 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":91,"threads_started":1}
I20260925 13:07:17.562414 2559 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:07:17.562552 2559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:07:17.563717 2696 tablet_service.cc:1467] Processing CreateTablet for tablet c05161c994f04223aeac8581d707c86b (DEFAULT_TABLE table=test-table [id=d515aa9b4e894baabc01cd2ca219920e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:17.563735 2836 tablet_service.cc:1467] Processing CreateTablet for tablet c05161c994f04223aeac8581d707c86b (DEFAULT_TABLE table=test-table [id=d515aa9b4e894baabc01cd2ca219920e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:17.563881 2836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c05161c994f04223aeac8581d707c86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:17.563877 2696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c05161c994f04223aeac8581d707c86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:17.564818 3162 tablet_bootstrap.cc:492] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f: Bootstrap starting.
I20260925 13:07:17.564855 3163 tablet_bootstrap.cc:492] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2: Bootstrap starting.
I20260925 13:07:17.565054 2959 tablet_service.cc:1467] Processing CreateTablet for tablet c05161c994f04223aeac8581d707c86b (DEFAULT_TABLE table=test-table [id=d515aa9b4e894baabc01cd2ca219920e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:17.565151 2959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c05161c994f04223aeac8581d707c86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:17.565215 3163 tablet_bootstrap.cc:654] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:17.565254 3162 tablet_bootstrap.cc:654] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:17.565774 3162 tablet_bootstrap.cc:492] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f: No bootstrap required, opened a new log
I20260925 13:07:17.565779 3163 tablet_bootstrap.cc:492] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2: No bootstrap required, opened a new log
I20260925 13:07:17.565815 3163 ts_tablet_manager.cc:1397] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:17.565821 3162 ts_tablet_manager.cc:1397] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:17.565900 3163 raft_consensus.cc:348] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:17.565945 3163 raft_consensus.cc:374] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:17.565958 3163 raft_consensus.cc:729] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a8bd366b904125a208127c85eddbc2, State: Initialized, Role: FOLLOWER
I20260925 13:07:17.565946 3162 raft_consensus.cc:348] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:17.566010 3162 raft_consensus.cc:374] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:17.566031 3162 raft_consensus.cc:729] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4286e8349aa04b22bd638c383898ae9f, State: Initialized, Role: FOLLOWER
I20260925 13:07:17.566017 3163 consensus_queue.cc:260] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:17.566082 3162 consensus_queue.cc:260] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:17.566108 3163 ts_tablet_manager.cc:1428] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:17.566236 3162 ts_tablet_manager.cc:1428] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:17.566525 3167 tablet_bootstrap.cc:492] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc: Bootstrap starting.
I20260925 13:07:17.566882 3167 tablet_bootstrap.cc:654] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:17.567406 3167 tablet_bootstrap.cc:492] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc: No bootstrap required, opened a new log
I20260925 13:07:17.567482 3167 ts_tablet_manager.cc:1397] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:17.567613 3167 raft_consensus.cc:348] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:17.567683 3167 raft_consensus.cc:374] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:17.567708 3167 raft_consensus.cc:729] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcca9f2896b045be991f4117081b10bc, State: Initialized, Role: FOLLOWER
I20260925 13:07:17.567771 3167 consensus_queue.cc:260] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:17.567868 3167 ts_tablet_manager.cc:1428] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:17.652755 3146 raft_consensus.cc:482] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:17.652851 3146 raft_consensus.cc:504] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:17.653004 3146 leader_election.cc:288] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4286e8349aa04b22bd638c383898ae9f (127.0.206.65:36595), fcca9f2896b045be991f4117081b10bc (127.0.206.67:44053)
I20260925 13:07:17.653157 2716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05161c994f04223aeac8581d707c86b" candidate_uuid: "01a8bd366b904125a208127c85eddbc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4286e8349aa04b22bd638c383898ae9f" is_pre_election: true
I20260925 13:07:17.653172 2996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05161c994f04223aeac8581d707c86b" candidate_uuid: "01a8bd366b904125a208127c85eddbc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcca9f2896b045be991f4117081b10bc" is_pre_election: true
I20260925 13:07:17.653255 2996 raft_consensus.cc:2454] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01a8bd366b904125a208127c85eddbc2 in term 0.
I20260925 13:07:17.653255 2716 raft_consensus.cc:2454] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01a8bd366b904125a208127c85eddbc2 in term 0.
I20260925 13:07:17.653429 2787 leader_election.cc:302] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a8bd366b904125a208127c85eddbc2, 4286e8349aa04b22bd638c383898ae9f; no voters:
I20260925 13:07:17.653529 3146 raft_consensus.cc:2781] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:07:17.653568 3146 raft_consensus.cc:482] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:07:17.653594 3146 raft_consensus.cc:3037] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:17.654217 3146 raft_consensus.cc:504] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:17.654347 3146 leader_election.cc:288] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [CANDIDATE]: Term 1 election: Requested vote from peers 4286e8349aa04b22bd638c383898ae9f (127.0.206.65:36595), fcca9f2896b045be991f4117081b10bc (127.0.206.67:44053)
I20260925 13:07:17.654502 2996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05161c994f04223aeac8581d707c86b" candidate_uuid: "01a8bd366b904125a208127c85eddbc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcca9f2896b045be991f4117081b10bc"
I20260925 13:07:17.654505 2716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05161c994f04223aeac8581d707c86b" candidate_uuid: "01a8bd366b904125a208127c85eddbc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4286e8349aa04b22bd638c383898ae9f"
I20260925 13:07:17.654592 2716 raft_consensus.cc:3037] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:17.654595 2996 raft_consensus.cc:3037] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:17.655251 2716 raft_consensus.cc:2454] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a8bd366b904125a208127c85eddbc2 in term 1.
I20260925 13:07:17.655268 2996 raft_consensus.cc:2454] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a8bd366b904125a208127c85eddbc2 in term 1.
I20260925 13:07:17.655433 2787 leader_election.cc:302] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a8bd366b904125a208127c85eddbc2, 4286e8349aa04b22bd638c383898ae9f; no voters:
I20260925 13:07:17.655558 3146 raft_consensus.cc:2781] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:17.655624 3146 raft_consensus.cc:686] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [term 1 LEADER]: Becoming Leader. State: Replica: 01a8bd366b904125a208127c85eddbc2, State: Running, Role: LEADER
I20260925 13:07:17.655673 3146 consensus_queue.cc:237] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:17.656224 2559 catalog_manager.cc:5495] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01a8bd366b904125a208127c85eddbc2 (127.0.206.66). New cstate: current_term: 1 leader_uuid: "01a8bd366b904125a208127c85eddbc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } health_report { overall_health: UNKNOWN } } }
I20260925 13:07:17.662825 3171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:17.660320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37814 (local address 127.0.206.66:43083)
0925 13:07:17.660429 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:17.660431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:17.660438 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:07:17.660455 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:17.660456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:17.660464 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:17.660519 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:17.660715 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:17.661311 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:17.662128 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:17.662336 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:17.662409 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:17.662556 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:17.662587 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:17.662640 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:17.662731 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 13:07:17.662750 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":67,"threads_started":1}
I20260925 13:07:17.666797 3171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:17.664942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:34181 (local address 127.0.206.66:43083)
0925 13:07:17.664973 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:17.664974 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:17.664983 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:17.665023 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:17.665024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:17.665031 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:17.665079 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:17.665180 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:17.665591 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:17.666255 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:17.666419 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:17.666465 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:17.666576 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:17.666595 (+ 19us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:17.666631 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:17.666682 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 13:07:17.666699 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 13:07:17.666859 3174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:17.664792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:43083 (local address 127.0.206.66:34181)
0925 13:07:17.664922 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:17.664932 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:17.664979 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:17.665095 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:17.665096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:17.665097 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 13:07:17.665127 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:17.665128 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:17.665649 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:17.665649 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:17.666192 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:17.666193 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:17.666481 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:17.666482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:17.666518 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:17.666522 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 13:07:17.666645 (+ 123us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 13:07:17.666647 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:07:17.666680 (+ 33us) client_negotiation.cc:240] Negotiation successful
0925 13:07:17.666757 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":57,"threads_started":1}
I20260925 13:07:17.668481 2996 raft_consensus.cc:1261] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc [term 1 FOLLOWER]: Refusing update from remote peer 01a8bd366b904125a208127c85eddbc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:17.668480 2716 raft_consensus.cc:1261] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f [term 1 FOLLOWER]: Refusing update from remote peer 01a8bd366b904125a208127c85eddbc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:17.668663 3129 consensus_queue.cc:1035] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:17.668751 3146 consensus_queue.cc:1035] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:17.678411 2559 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:17.678493 2559 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:17.679030 2559 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9: Sending DeleteTablet for 3 replicas of tablet c05161c994f04223aeac8581d707c86b
I20260925 13:07:17.679258 2696 tablet_service.cc:1514] Processing DeleteTablet for tablet c05161c994f04223aeac8581d707c86b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:17 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:58568
I20260925 13:07:17.679276 2836 tablet_service.cc:1514] Processing DeleteTablet for tablet c05161c994f04223aeac8581d707c86b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:17 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:37798
I20260925 13:07:17.679351 3152 tablet_replica.cc:330] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f: stopping tablet replica
I20260925 13:07:17.679370 3151 tablet_replica.cc:330] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2: stopping tablet replica
I20260925 13:07:17.679411 3152 raft_consensus.cc:2229] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:17.679436 3151 raft_consensus.cc:2229] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:07:17.679456 3152 raft_consensus.cc:2258] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:17.679520 3151 raft_consensus.cc:2258] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:17.679757 3152 ts_tablet_manager.cc:1905] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:17.679768 2959 tablet_service.cc:1514] Processing DeleteTablet for tablet c05161c994f04223aeac8581d707c86b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:17 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:56266
I20260925 13:07:17.679834 3153 tablet_replica.cc:330] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc: stopping tablet replica
I20260925 13:07:17.679883 3153 raft_consensus.cc:2229] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:17.679922 3153 raft_consensus.cc:2258] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:17.680121 2559 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:07:17.680187 3153 ts_tablet_manager.cc:1905] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:07:17.680233 2559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:07:17.681427 3152 ts_tablet_manager.cc:1918] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 13:07:17.681486 3152 log.cc:1196] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-0/wal/wals/c05161c994f04223aeac8581d707c86b
I20260925 13:07:17.681582 3153 ts_tablet_manager.cc:1918] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 13:07:17.681591 2696 tablet_service.cc:1467] Processing CreateTablet for tablet e9ed2fdc8f1541699a7468524f612860 (DEFAULT_TABLE table=test-table [id=05e24c7f514045d3a40f12d1babb395c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:17.681630 3153 log.cc:1196] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/c05161c994f04223aeac8581d707c86b
I20260925 13:07:17.681677 2696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9ed2fdc8f1541699a7468524f612860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:17.681708 3152 ts_tablet_manager.cc:1939] T c05161c994f04223aeac8581d707c86b P 4286e8349aa04b22bd638c383898ae9f: Deleting consensus metadata
I20260925 13:07:17.681842 3153 ts_tablet_manager.cc:1939] T c05161c994f04223aeac8581d707c86b P fcca9f2896b045be991f4117081b10bc: Deleting consensus metadata
I20260925 13:07:17.682654 3162 tablet_bootstrap.cc:492] T e9ed2fdc8f1541699a7468524f612860 P 4286e8349aa04b22bd638c383898ae9f: Bootstrap starting.
I20260925 13:07:17.682760 2543 catalog_manager.cc:4841] TS 4286e8349aa04b22bd638c383898ae9f (127.0.206.65:36595): tablet c05161c994f04223aeac8581d707c86b (table test-table [id=d515aa9b4e894baabc01cd2ca219920e]) successfully deleted
I20260925 13:07:17.682874 2546 catalog_manager.cc:4841] TS fcca9f2896b045be991f4117081b10bc (127.0.206.67:44053): tablet c05161c994f04223aeac8581d707c86b (table test-table [id=d515aa9b4e894baabc01cd2ca219920e]) successfully deleted
I20260925 13:07:17.682986 3162 tablet_bootstrap.cc:654] T e9ed2fdc8f1541699a7468524f612860 P 4286e8349aa04b22bd638c383898ae9f: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:17.682976 2959 tablet_service.cc:1467] Processing CreateTablet for tablet e9ed2fdc8f1541699a7468524f612860 (DEFAULT_TABLE table=test-table [id=05e24c7f514045d3a40f12d1babb395c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:17.683060 2959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9ed2fdc8f1541699a7468524f612860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:17.683516 3162 tablet_bootstrap.cc:492] T e9ed2fdc8f1541699a7468524f612860 P 4286e8349aa04b22bd638c383898ae9f: No bootstrap required, opened a new log
I20260925 13:07:17.683559 3162 ts_tablet_manager.cc:1397] T e9ed2fdc8f1541699a7468524f612860 P 4286e8349aa04b22bd638c383898ae9f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:17.683661 3162 raft_consensus.cc:348] T e9ed2fdc8f1541699a7468524f612860 P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:17.683725 3162 raft_consensus.cc:374] T e9ed2fdc8f1541699a7468524f612860 P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:17.683748 3162 raft_consensus.cc:729] T e9ed2fdc8f1541699a7468524f612860 P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4286e8349aa04b22bd638c383898ae9f, State: Initialized, Role: FOLLOWER
I20260925 13:07:17.683789 3162 consensus_queue.cc:260] T e9ed2fdc8f1541699a7468524f612860 P 4286e8349aa04b22bd638c383898ae9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:17.683876 3162 ts_tablet_manager.cc:1428] T e9ed2fdc8f1541699a7468524f612860 P 4286e8349aa04b22bd638c383898ae9f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:17.684221 3167 tablet_bootstrap.cc:492] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc: Bootstrap starting.
I20260925 13:07:17.684252 2836 tablet_service.cc:1467] Processing CreateTablet for tablet e9ed2fdc8f1541699a7468524f612860 (DEFAULT_TABLE table=test-table [id=05e24c7f514045d3a40f12d1babb395c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:17.684371 2836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9ed2fdc8f1541699a7468524f612860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:17.684516 3167 tablet_bootstrap.cc:654] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:17.685070 3167 tablet_bootstrap.cc:492] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc: No bootstrap required, opened a new log
I20260925 13:07:17.685110 3167 ts_tablet_manager.cc:1397] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:17.685194 3167 raft_consensus.cc:348] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:17.685248 3167 raft_consensus.cc:374] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:17.685268 3167 raft_consensus.cc:729] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcca9f2896b045be991f4117081b10bc, State: Initialized, Role: FOLLOWER
I20260925 13:07:17.685309 3167 consensus_queue.cc:260] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:17.685348 3163 tablet_bootstrap.cc:492] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2: Bootstrap starting.
I20260925 13:07:17.685411 3167 ts_tablet_manager.cc:1428] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:17.685659 3163 tablet_bootstrap.cc:654] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:17.686267 3163 tablet_bootstrap.cc:492] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2: No bootstrap required, opened a new log
I20260925 13:07:17.686316 3163 ts_tablet_manager.cc:1397] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:17.686426 3163 raft_consensus.cc:348] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:17.686489 3163 raft_consensus.cc:374] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:17.686512 3163 raft_consensus.cc:729] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a8bd366b904125a208127c85eddbc2, State: Initialized, Role: FOLLOWER
I20260925 13:07:17.686559 3163 consensus_queue.cc:260] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:17.686635 3151 ts_tablet_manager.cc:1905] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:17.686658 3163 ts_tablet_manager.cc:1428] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:17.687968 3151 ts_tablet_manager.cc:1918] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 13:07:17.688024 3151 log.cc:1196] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/c05161c994f04223aeac8581d707c86b
I20260925 13:07:17.688220 3151 ts_tablet_manager.cc:1939] T c05161c994f04223aeac8581d707c86b P 01a8bd366b904125a208127c85eddbc2: Deleting consensus metadata
I20260925 13:07:17.688459 2543 catalog_manager.cc:4841] TS 01a8bd366b904125a208127c85eddbc2 (127.0.206.66:43083): tablet c05161c994f04223aeac8581d707c86b (table test-table [id=d515aa9b4e894baabc01cd2ca219920e]) successfully deleted
I20260925 13:07:18.036602 3144 raft_consensus.cc:482] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:18.036687 3144 raft_consensus.cc:504] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:18.036909 3144 leader_election.cc:288] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4286e8349aa04b22bd638c383898ae9f (127.0.206.65:36595), 01a8bd366b904125a208127c85eddbc2 (127.0.206.66:43083)
I20260925 13:07:18.037147 2716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ed2fdc8f1541699a7468524f612860" candidate_uuid: "fcca9f2896b045be991f4117081b10bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4286e8349aa04b22bd638c383898ae9f" is_pre_election: true
I20260925 13:07:18.037160 2856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ed2fdc8f1541699a7468524f612860" candidate_uuid: "fcca9f2896b045be991f4117081b10bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a8bd366b904125a208127c85eddbc2" is_pre_election: true
I20260925 13:07:18.037245 2716 raft_consensus.cc:2454] T e9ed2fdc8f1541699a7468524f612860 P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcca9f2896b045be991f4117081b10bc in term 0.
I20260925 13:07:18.037245 2856 raft_consensus.cc:2454] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcca9f2896b045be991f4117081b10bc in term 0.
I20260925 13:07:18.037407 2927 leader_election.cc:302] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a8bd366b904125a208127c85eddbc2, fcca9f2896b045be991f4117081b10bc; no voters:
I20260925 13:07:18.037578 3144 raft_consensus.cc:2781] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:07:18.037642 3144 raft_consensus.cc:482] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:07:18.037674 3144 raft_consensus.cc:3037] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:18.038326 3144 raft_consensus.cc:504] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:18.038458 3144 leader_election.cc:288] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [CANDIDATE]: Term 1 election: Requested vote from peers 4286e8349aa04b22bd638c383898ae9f (127.0.206.65:36595), 01a8bd366b904125a208127c85eddbc2 (127.0.206.66:43083)
I20260925 13:07:18.038640 2716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ed2fdc8f1541699a7468524f612860" candidate_uuid: "fcca9f2896b045be991f4117081b10bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4286e8349aa04b22bd638c383898ae9f"
I20260925 13:07:18.038669 2856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9ed2fdc8f1541699a7468524f612860" candidate_uuid: "fcca9f2896b045be991f4117081b10bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a8bd366b904125a208127c85eddbc2"
I20260925 13:07:18.038755 2716 raft_consensus.cc:3037] T e9ed2fdc8f1541699a7468524f612860 P 4286e8349aa04b22bd638c383898ae9f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:18.038763 2856 raft_consensus.cc:3037] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:18.039438 2856 raft_consensus.cc:2454] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcca9f2896b045be991f4117081b10bc in term 1.
I20260925 13:07:18.039458 2716 raft_consensus.cc:2454] T e9ed2fdc8f1541699a7468524f612860 P 4286e8349aa04b22bd638c383898ae9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcca9f2896b045be991f4117081b10bc in term 1.
I20260925 13:07:18.039602 2927 leader_election.cc:302] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a8bd366b904125a208127c85eddbc2, fcca9f2896b045be991f4117081b10bc; no voters:
I20260925 13:07:18.039702 3144 raft_consensus.cc:2781] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:18.039799 3144 raft_consensus.cc:686] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [term 1 LEADER]: Becoming Leader. State: Replica: fcca9f2896b045be991f4117081b10bc, State: Running, Role: LEADER
I20260925 13:07:18.039860 3144 consensus_queue.cc:237] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } }
I20260925 13:07:18.040369 2559 catalog_manager.cc:5495] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc reported cstate change: term changed from 0 to 1, leader changed from <none> to fcca9f2896b045be991f4117081b10bc (127.0.206.67). New cstate: current_term: 1 leader_uuid: "fcca9f2896b045be991f4117081b10bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcca9f2896b045be991f4117081b10bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44053 } health_report { overall_health: HEALTHY } } }
I20260925 13:07:18.130606 3183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:18.128403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56292 (local address 127.0.206.67:44053)
0925 13:07:18.128541 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:18.128543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:18.128551 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:18.128579 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:18.128580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:18.128590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:18.128654 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:18.128755 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:18.129227 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:18.129967 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:18.130171 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:18.130233 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:18.130397 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:18.130423 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:18.130464 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:18.130519 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 13:07:18.130535 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260925 13:07:18.132758 2856 raft_consensus.cc:1261] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2 [term 1 FOLLOWER]: Refusing update from remote peer fcca9f2896b045be991f4117081b10bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:18.132761 2716 raft_consensus.cc:1261] T e9ed2fdc8f1541699a7468524f612860 P 4286e8349aa04b22bd638c383898ae9f [term 1 FOLLOWER]: Refusing update from remote peer fcca9f2896b045be991f4117081b10bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:18.132979 3181 consensus_queue.cc:1035] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4286e8349aa04b22bd638c383898ae9f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:18.133080 3144 consensus_queue.cc:1035] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a8bd366b904125a208127c85eddbc2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 13:07:18.137217 3134 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 13:07:19.654881 2559 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:19.654975 2559 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:19.655721 2559 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 29e5d573cd3246bd9622d7574658e3f9: Sending DeleteTablet for 3 replicas of tablet e9ed2fdc8f1541699a7468524f612860
I20260925 13:07:19.655983 2836 tablet_service.cc:1514] Processing DeleteTablet for tablet e9ed2fdc8f1541699a7468524f612860 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:19 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:37798
I20260925 13:07:19.655970 2959 tablet_service.cc:1514] Processing DeleteTablet for tablet e9ed2fdc8f1541699a7468524f612860 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:19 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:56266
I20260925 13:07:19.656107 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 2631
I20260925 13:07:19.656143 2696 tablet_service.cc:1514] Processing DeleteTablet for tablet e9ed2fdc8f1541699a7468524f612860 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:19 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:58568
I20260925 13:07:19.657732 3212 tablet_replica.cc:330] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2: stopping tablet replica
I20260925 13:07:19.657773 3212 raft_consensus.cc:2229] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:19.657797 3212 raft_consensus.cc:2258] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:19.657904 3211 tablet_replica.cc:330] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc: stopping tablet replica
I20260925 13:07:19.657954 3211 raft_consensus.cc:2229] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:07:19.658007 3211 raft_consensus.cc:2258] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:19.658077 3212 ts_tablet_manager.cc:1905] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:19.658208 3211 ts_tablet_manager.cc:1905] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:19.659868 3211 ts_tablet_manager.cc:1918] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 13:07:19.659868 3212 ts_tablet_manager.cc:1918] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 13:07:19.659920 3211 log.cc:1196] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/e9ed2fdc8f1541699a7468524f612860
I20260925 13:07:19.659926 3212 log.cc:1196] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/e9ed2fdc8f1541699a7468524f612860
I20260925 13:07:19.660163 3212 ts_tablet_manager.cc:1939] T e9ed2fdc8f1541699a7468524f612860 P 01a8bd366b904125a208127c85eddbc2: Deleting consensus metadata
I20260925 13:07:19.660203 3211 ts_tablet_manager.cc:1939] T e9ed2fdc8f1541699a7468524f612860 P fcca9f2896b045be991f4117081b10bc: Deleting consensus metadata
I20260925 13:07:19.660488 2543 catalog_manager.cc:4841] TS 01a8bd366b904125a208127c85eddbc2 (127.0.206.66:43083): tablet e9ed2fdc8f1541699a7468524f612860 (table test-table [id=05e24c7f514045d3a40f12d1babb395c]) successfully deleted
I20260925 13:07:19.660506 2546 catalog_manager.cc:4841] TS fcca9f2896b045be991f4117081b10bc (127.0.206.67:44053): tablet e9ed2fdc8f1541699a7468524f612860 (table test-table [id=05e24c7f514045d3a40f12d1babb395c]) successfully deleted
W20260925 13:07:19.662230 2543 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:36595 (error 108)
I20260925 13:07:19.662489 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 2771
I20260925 13:07:19.666937 3214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:19.662535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:36595 (local address 127.0.0.1:58594)
0925 13:07:19.666844 (+ 4309us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:19.666900 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4287,"thread_start_us":71,"threads_started":1}
W20260925 13:07:19.667091 2543 catalog_manager.cc:4584] TS 4286e8349aa04b22bd638c383898ae9f (127.0.206.65:36595): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e9ed2fdc8f1541699a7468524f612860: Network error: Client connection negotiation failed: client connection to 127.0.206.65:36595: connect: Connection refused (error 111)
I20260925 13:07:19.668752 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 2911
I20260925 13:07:19.673768 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 2527
2026-09-25T13:07:19Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5976 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:07:19 dist-test-slave-chkf krb5kdc[3217](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:07:19 dist-test-slave-chkf krb5kdc[3217](info): set up 2 sockets
Sep 25 13:07:19 dist-test-slave-chkf krb5kdc[3217](info): commencing operation
krb5kdc: starting...
W20260925 13:07:20.710533 825 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:07:20 dist-test-slave-chkf krb5kdc[3217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:07:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:07:20.821427 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:41971
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:41971
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:20.900092 3230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:20.900278 3230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:20.900301 3230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:20.901474 3230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:20.901515 3230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:20.901528 3230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:20.901541 3230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:20.901594 3230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:07:20.903088 3230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:41971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:41971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:20.903323 3230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:20.903539 3230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:20.908170 3236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:20.908218 3238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:20.908260 3230 server_base.cc:1034] running on GCE node
W20260925 13:07:20.908202 3235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:20.908607 3230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:20.908907 3230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:20.910073 3230 hybrid_clock.cc:648] HybridClock initialized: now 1790341640910052 us; error 33 us; skew 500 ppm
Sep 25 13:07:20 dist-test-slave-chkf krb5kdc[3217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341640, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:20.912330 3230 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:07:20.912753 3230 webserver.cc:460] Webserver started at http://127.0.206.126:39629/ using document root <none> and password file <none>
I20260925 13:07:20.912971 3230 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:20.913036 3230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:20.913177 3230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:20.914131 3230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "73b209869ec14c5ab7f667a95b2b7596"
format_stamp: "Formatted at 2026-09-25 13:07:20 on dist-test-slave-chkf"
server_key: "7b5fb5631514b3a1e9fa03fd987b0f40"
server_key_iv: "e527b92d931b8ce738afd8eb9e868c49"
server_key_version: "encryptionkey@0"
I20260925 13:07:20.914476 3230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "73b209869ec14c5ab7f667a95b2b7596"
format_stamp: "Formatted at 2026-09-25 13:07:20 on dist-test-slave-chkf"
server_key: "7b5fb5631514b3a1e9fa03fd987b0f40"
server_key_iv: "e527b92d931b8ce738afd8eb9e868c49"
server_key_version: "encryptionkey@0"
I20260925 13:07:20.915807 3230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:07:20.916468 3245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:20.916636 3230 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:20.916729 3230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "73b209869ec14c5ab7f667a95b2b7596"
format_stamp: "Formatted at 2026-09-25 13:07:20 on dist-test-slave-chkf"
server_key: "7b5fb5631514b3a1e9fa03fd987b0f40"
server_key_iv: "e527b92d931b8ce738afd8eb9e868c49"
server_key_version: "encryptionkey@0"
I20260925 13:07:20.916810 3230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:20.926457 3230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:20.928161 3230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:20.928309 3230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:20.932261 3230 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:41971
I20260925 13:07:20.932291 3297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:41971 every 8 connection(s)
I20260925 13:07:20.932739 3230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:20.933326 3298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:20.935438 3298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596: Bootstrap starting.
I20260925 13:07:20.935590 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 3230
I20260925 13:07:20.935690 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:07:20.935859 825 external_mini_cluster.cc:1428] Setting key 51759f493f3e998bc3d029d7b251256a
I20260925 13:07:20.936178 3298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:20.936530 3298 log.cc:824] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:20.937260 3298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596: No bootstrap required, opened a new log
I20260925 13:07:20.939492 3298 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b209869ec14c5ab7f667a95b2b7596" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41971 } }
I20260925 13:07:20.939647 3298 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:20.939671 3298 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73b209869ec14c5ab7f667a95b2b7596, State: Initialized, Role: FOLLOWER
I20260925 13:07:20.939808 3298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b209869ec14c5ab7f667a95b2b7596" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41971 } }
I20260925 13:07:20.939881 3298 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:20.939903 3298 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:20.939975 3298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:20.940542 3298 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b209869ec14c5ab7f667a95b2b7596" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41971 } }
I20260925 13:07:20.940641 3298 leader_election.cc:302] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73b209869ec14c5ab7f667a95b2b7596; no voters:
I20260925 13:07:20.940781 3298 leader_election.cc:288] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:07:20.940843 3303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:20.941015 3298 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:20.941035 3303 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596 [term 1 LEADER]: Becoming Leader. State: Replica: 73b209869ec14c5ab7f667a95b2b7596, State: Running, Role: LEADER
I20260925 13:07:20.941257 3303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b209869ec14c5ab7f667a95b2b7596" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41971 } }
I20260925 13:07:20.941604 3304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73b209869ec14c5ab7f667a95b2b7596" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b209869ec14c5ab7f667a95b2b7596" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41971 } } }
I20260925 13:07:20.941686 3304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:20.941926 3305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73b209869ec14c5ab7f667a95b2b7596. Latest consensus state: current_term: 1 leader_uuid: "73b209869ec14c5ab7f667a95b2b7596" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73b209869ec14c5ab7f667a95b2b7596" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41971 } } }
I20260925 13:07:20.942058 3305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:20.942363 3312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:20.942989 3312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 13:07:20 dist-test-slave-chkf krb5kdc[3217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:20.944486 3312 catalog_manager.cc:1347] Generated new cluster ID: 3bea7ec741934557b2329cb57c7ebbeb
I20260925 13:07:20.944537 3312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:07:20.949566 3312 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:07:20.950096 3312 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:20.950105 3301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:20.936384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33696 (local address 127.0.206.126:41971)
0925 13:07:20.936561 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:20.936569 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:20.936600 (+ 31us) server_negotiation.cc:407] Connection header received
0925 13:07:20.937135 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:20.937143 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:20.937200 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:20.937346 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:20.937962 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:20.939408 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:20.940355 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:20.940970 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:20.941577 (+ 607us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:20.944122 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:20.944125 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:20.944125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:20.944133 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:20.948338 (+ 4205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:20.948727 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:20.948730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:20.948731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:20.948836 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:20.949049 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:20.949050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:20.949051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:20.949319 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:20.949483 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:20.949673 (+ 190us) server_negotiation.cc:299] Negotiation successful
0925 13:07:20.949911 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
I20260925 13:07:20.953961 3312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73b209869ec14c5ab7f667a95b2b7596: Generated new TSK 0
I20260925 13:07:20.954123 3312 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260925 13:07:20.981490 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:41971
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:21.061555 3324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:21.061748 3324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:21.061769 3324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:21.062954 3324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:21.062992 3324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:21.063019 3324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:07:21.064469 3324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:41971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:21.064710 3324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:21.064939 3324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:21.070053 3330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:21.070025 3329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:21.070170 3324 server_base.cc:1034] running on GCE node
W20260925 13:07:21.070134 3332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:21.070447 3324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:21.070642 3324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:21.071776 3324 hybrid_clock.cc:648] HybridClock initialized: now 1790341641071741 us; error 52 us; skew 500 ppm
Sep 25 13:07:21 dist-test-slave-chkf krb5kdc[3217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341641, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:21.074180 3324 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:07:21.074682 3324 webserver.cc:460] Webserver started at http://127.0.206.65:41297/ using document root <none> and password file <none>
I20260925 13:07:21.074970 3324 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:21.075014 3324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:21.075160 3324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:21.076161 3324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "6b9075a4e0b449eda39c0973448a17b6"
format_stamp: "Formatted at 2026-09-25 13:07:21 on dist-test-slave-chkf"
server_key: "20d478f48a110fc713d26d27bdd93250"
server_key_iv: "31ce4f73b99fb70f9a154101380626d3"
server_key_version: "encryptionkey@0"
I20260925 13:07:21.076507 3324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "6b9075a4e0b449eda39c0973448a17b6"
format_stamp: "Formatted at 2026-09-25 13:07:21 on dist-test-slave-chkf"
server_key: "20d478f48a110fc713d26d27bdd93250"
server_key_iv: "31ce4f73b99fb70f9a154101380626d3"
server_key_version: "encryptionkey@0"
I20260925 13:07:21.077749 3324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:07:21.078449 3339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:21.078615 3324 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:21.078696 3324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "6b9075a4e0b449eda39c0973448a17b6"
format_stamp: "Formatted at 2026-09-25 13:07:21 on dist-test-slave-chkf"
server_key: "20d478f48a110fc713d26d27bdd93250"
server_key_iv: "31ce4f73b99fb70f9a154101380626d3"
server_key_version: "encryptionkey@0"
I20260925 13:07:21.078814 3324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:21.096306 3324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:21.097939 3324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:21.098090 3324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:21.098289 3324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:21.098580 3324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:21.098616 3324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:21.098635 3324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:21.098723 3324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:21.104106 3324 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:43235
I20260925 13:07:21.104140 3452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:43235 every 8 connection(s)
I20260925 13:07:21.104449 3324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:07:21.105702 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 3324
I20260925 13:07:21.105803 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:07:21.105991 825 external_mini_cluster.cc:1428] Setting key 0afe52dea03b25ed39f8470d97f3187a
Sep 25 13:07:21 dist-test-slave-chkf krb5kdc[3217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341641, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
I20260925 13:07:21.114939 3301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:21.105184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:39179 (local address 127.0.206.126:41971)
0925 13:07:21.105257 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:21.105259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:21.105417 (+ 158us) server_negotiation.cc:407] Connection header received
0925 13:07:21.106436 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:21.106437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:21.106448 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:21.106507 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:21.107210 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:21.107998 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:21.109492 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:21.109694 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:21.109752 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:21.112089 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:21.112090 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:21.112090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:21.112098 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:21.113509 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:21.113923 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:21.113924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:21.113925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:21.114009 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:21.114250 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:21.114251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:21.114251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:21.114363 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:21.114453 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:21.114728 (+ 275us) server_negotiation.cc:299] Negotiation successful
0925 13:07:21.114849 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 13:07:21.115092 3455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:21.105130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41971 (local address 127.0.206.65:39179)
0925 13:07:21.105306 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:21.105341 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:21.106322 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:21.106606 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:21.106610 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:21.106944 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:21.107120 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:21.107123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:21.108096 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:21.108098 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:21.109407 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:21.109409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:21.109780 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:21.109782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:21.109921 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:21.110465 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:21.110476 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:21.111992 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:21.113610 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:21.113616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:21.113618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:21.113835 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:21.114072 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:21.114072 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:21.114073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:21.114173 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:21.114446 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:21.114452 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:21.114585 (+ 133us) client_negotiation.cc:769] Sending connection context
0925 13:07:21.114692 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 13:07:21.114859 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":93,"threads_started":1}
I20260925 13:07:21.115640 3453 heartbeater.cc:347] Connected to a master server at 127.0.206.126:41971
I20260925 13:07:21.115734 3453 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:21.115885 3453 heartbeater.cc:511] Master 127.0.206.126:41971 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
I20260925 13:07:21.116415 3262 ts_manager.cc:194] Registered new tserver with Master: 6b9075a4e0b449eda39c0973448a17b6 (127.0.206.65:43235)
I20260925 13:07:21.117337 3262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:39179
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260925 13:07:21.131464 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:41971
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:21.210323 3458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:21.210551 3458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:21.210588 3458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:21.211731 3458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:21.211800 3458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:21.211845 3458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:07:21.213312 3458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:41971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:21.213573 3458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:21.213793 3458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:21.218611 3466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:21.218592 3463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:21.218592 3464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:21.218765 3458 server_base.cc:1034] running on GCE node
I20260925 13:07:21.219164 3458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:21.219405 3458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:21.220584 3458 hybrid_clock.cc:648] HybridClock initialized: now 1790341641220575 us; error 27 us; skew 500 ppm
Sep 25 13:07:21 dist-test-slave-chkf krb5kdc[3217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341641, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:21.222944 3458 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:07:21.223346 3458 webserver.cc:460] Webserver started at http://127.0.206.66:36881/ using document root <none> and password file <none>
I20260925 13:07:21.223572 3458 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:21.223639 3458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:21.223767 3458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:21.224727 3458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "c0c17fd1640c4e83a31b052e35b92379"
format_stamp: "Formatted at 2026-09-25 13:07:21 on dist-test-slave-chkf"
server_key: "f83ff308664640d58bddc3ee503ea488"
server_key_iv: "662887399c05f2de8293894a428fe2b6"
server_key_version: "encryptionkey@0"
I20260925 13:07:21.225096 3458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "c0c17fd1640c4e83a31b052e35b92379"
format_stamp: "Formatted at 2026-09-25 13:07:21 on dist-test-slave-chkf"
server_key: "f83ff308664640d58bddc3ee503ea488"
server_key_iv: "662887399c05f2de8293894a428fe2b6"
server_key_version: "encryptionkey@0"
I20260925 13:07:21.226356 3458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:07:21.227083 3473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:21.227241 3458 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:21.227339 3458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "c0c17fd1640c4e83a31b052e35b92379"
format_stamp: "Formatted at 2026-09-25 13:07:21 on dist-test-slave-chkf"
server_key: "f83ff308664640d58bddc3ee503ea488"
server_key_iv: "662887399c05f2de8293894a428fe2b6"
server_key_version: "encryptionkey@0"
I20260925 13:07:21.227412 3458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:21.240571 3458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:21.242453 3458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:21.242606 3458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:21.242854 3458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:21.243170 3458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:21.243229 3458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:21.243264 3458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:21.243291 3458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:21.248832 3458 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:33925
I20260925 13:07:21.248878 3586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:33925 every 8 connection(s)
I20260925 13:07:21.249164 3458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
Sep 25 13:07:21 dist-test-slave-chkf krb5kdc[3217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341641, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:21.256174 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 3458
I20260925 13:07:21.256256 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:07:21.256438 825 external_mini_cluster.cc:1428] Setting key d215d9224c6c6affa1f7e9c47a148ea2
I20260925 13:07:21.257699 3301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:21.249781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:45413 (local address 127.0.206.126:41971)
0925 13:07:21.249838 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:21.249840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:21.250022 (+ 182us) server_negotiation.cc:407] Connection header received
0925 13:07:21.250981 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:21.250982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:21.250993 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:21.251040 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:21.251571 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:21.252048 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:21.252948 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:21.253127 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:21.253175 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:21.255075 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:21.255076 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:21.255076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:21.255083 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:21.256244 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:21.256722 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:21.256723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:21.256724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:21.256788 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:21.257067 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:21.257068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:21.257068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:21.257183 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:21.257257 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:21.257530 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 13:07:21.257613 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 13:07:21.257887 3589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:21.249732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41971 (local address 127.0.206.66:45413)
0925 13:07:21.249906 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:21.249939 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:21.250885 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:21.251098 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:21.251101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:21.251322 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:21.251501 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:21.251505 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:21.252123 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:21.252124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:21.252877 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:21.252879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:21.253195 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:21.253196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:21.253286 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:21.253690 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:21.253697 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:21.254995 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:21.256341 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:21.256346 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:21.256347 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:21.256626 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:21.256869 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:21.256870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:21.256870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:21.256997 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:21.257261 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:21.257267 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:21.257395 (+ 128us) client_negotiation.cc:769] Sending connection context
0925 13:07:21.257502 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 13:07:21.257648 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":61,"threads_started":1}
I20260925 13:07:21.258319 3587 heartbeater.cc:347] Connected to a master server at 127.0.206.126:41971
I20260925 13:07:21.258423 3587 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:21.258607 3587 heartbeater.cc:511] Master 127.0.206.126:41971 requested a full tablet report, sending...
I20260925 13:07:21.259017 3262 ts_manager.cc:194] Registered new tserver with Master: c0c17fd1640c4e83a31b052e35b92379 (127.0.206.66:33925)
I20260925 13:07:21.259532 3262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:45413
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260925 13:07:21.281551 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:41971
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:21.360890 3592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:21.361081 3592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:21.361104 3592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:21.362246 3592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:21.362287 3592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:21.362313 3592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:07:21.363798 3592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:41971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:21.364038 3592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:21.364259 3592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:21.369088 3597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:21.369155 3598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:21.369093 3600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:21.369339 3592 server_base.cc:1034] running on GCE node
I20260925 13:07:21.369540 3592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:21.369781 3592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:21.370911 3592 hybrid_clock.cc:648] HybridClock initialized: now 1790341641370901 us; error 28 us; skew 500 ppm
Sep 25 13:07:21 dist-test-slave-chkf krb5kdc[3217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341641, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:21.373314 3592 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:07:21.373768 3592 webserver.cc:460] Webserver started at http://127.0.206.67:34601/ using document root <none> and password file <none>
I20260925 13:07:21.373986 3592 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:21.374032 3592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:21.374171 3592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:21.375226 3592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "abc67df1f7574c02b01f46cb3431e71a"
format_stamp: "Formatted at 2026-09-25 13:07:21 on dist-test-slave-chkf"
server_key: "6994051bb47c9ad919b67c5d2fa1d44c"
server_key_iv: "5f2f69d6d6c70555100a95703a32b3af"
server_key_version: "encryptionkey@0"
I20260925 13:07:21.375587 3592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "abc67df1f7574c02b01f46cb3431e71a"
format_stamp: "Formatted at 2026-09-25 13:07:21 on dist-test-slave-chkf"
server_key: "6994051bb47c9ad919b67c5d2fa1d44c"
server_key_iv: "5f2f69d6d6c70555100a95703a32b3af"
server_key_version: "encryptionkey@0"
I20260925 13:07:21.376842 3592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:07:21.377503 3607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:21.377672 3592 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:21.377774 3592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "abc67df1f7574c02b01f46cb3431e71a"
format_stamp: "Formatted at 2026-09-25 13:07:21 on dist-test-slave-chkf"
server_key: "6994051bb47c9ad919b67c5d2fa1d44c"
server_key_iv: "5f2f69d6d6c70555100a95703a32b3af"
server_key_version: "encryptionkey@0"
I20260925 13:07:21.377851 3592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:21.390784 3592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:21.392525 3592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:21.392681 3592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:21.392890 3592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:21.393189 3592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:21.393244 3592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:21.393285 3592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:21.393312 3592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:21.399153 3592 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:46507
I20260925 13:07:21.399199 3720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:46507 every 8 connection(s)
I20260925 13:07:21.399509 3592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
Sep 25 13:07:21 dist-test-slave-chkf krb5kdc[3217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341641, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:21.406235 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 3592
I20260925 13:07:21.406330 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:07:21.406589 825 external_mini_cluster.cc:1428] Setting key 43be2f319e56b0f3339c5677058bfe66
I20260925 13:07:21.408524 3301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:21.400189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:44329 (local address 127.0.206.126:41971)
0925 13:07:21.400231 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:21.400233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:21.400439 (+ 206us) server_negotiation.cc:407] Connection header received
0925 13:07:21.401306 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:21.401307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:21.401317 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:21.401374 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:21.401918 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:21.402401 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:21.403319 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:21.403487 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:21.403533 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:21.405483 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:21.405484 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:21.405485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:21.405491 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:21.407252 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:21.407702 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:21.407703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:21.407703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:21.407765 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:21.408005 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:21.408006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:21.408006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:21.408124 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:21.408186 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:21.408402 (+ 216us) server_negotiation.cc:299] Negotiation successful
0925 13:07:21.408462 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 13:07:21.408718 3723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:21.400174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41971 (local address 127.0.206.67:44329)
0925 13:07:21.400309 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:21.400343 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:21.401208 (+ 865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:21.401429 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:21.401432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:21.401663 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:21.401839 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:21.401843 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:21.402476 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:21.402476 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:21.403250 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:21.403252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:21.403552 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:21.403553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:21.403642 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:21.404051 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:21.404059 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:21.405382 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:21.407329 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:21.407335 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:21.407337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:21.407641 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:21.407823 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:21.407823 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:21.407824 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:21.407916 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:21.408186 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:21.408190 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:21.408292 (+ 102us) client_negotiation.cc:769] Sending connection context
0925 13:07:21.408371 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 13:07:21.408490 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":82,"threads_started":1}
I20260925 13:07:21.409185 3721 heartbeater.cc:347] Connected to a master server at 127.0.206.126:41971
I20260925 13:07:21.409291 3721 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:21.409463 3721 heartbeater.cc:511] Master 127.0.206.126:41971 requested a full tablet report, sending...
I20260925 13:07:21.409830 3262 ts_manager.cc:194] Registered new tserver with Master: abc67df1f7574c02b01f46cb3431e71a (127.0.206.67:46507)
I20260925 13:07:21.410218 3262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:44329
I20260925 13:07:21.418311 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 13:07:21.428999 3301 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:21.428187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33710 (local address 127.0.206.126:41971)
0925 13:07:21.428247 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:21.428248 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:21.428369 (+ 121us) server_negotiation.cc:407] Connection header received
0925 13:07:21.428423 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:21.428423 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:21.428432 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:21.428483 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:21.428921 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33710: BlockingRecv error: recv got EOF from 127.0.0.1:33710 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 13:07:21.430039 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 3324
I20260925 13:07:21.434208 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 3458
I20260925 13:07:21.439431 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 3592
I20260925 13:07:21.444780 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 3230
2026-09-25T13:07:21Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1770 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:07:21 dist-test-slave-chkf krb5kdc[3733](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:07:21 dist-test-slave-chkf krb5kdc[3733](info): set up 2 sockets
Sep 25 13:07:21 dist-test-slave-chkf krb5kdc[3733](info): commencing operation
krb5kdc: starting...
W20260925 13:07:22.482458 825 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:07:22 dist-test-slave-chkf krb5kdc[3733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:07:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:07:22.599676 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:37551
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:37551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:22.679023 3746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:22.679237 3746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:22.679275 3746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:22.680401 3746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:22.680470 3746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:22.680498 3746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:22.680528 3746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:22.680554 3746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:07:22.682003 3746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:37551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:37551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:22.682264 3746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:22.682482 3746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:22.687139 3754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:22.687146 3751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:22.687175 3746 server_base.cc:1034] running on GCE node
W20260925 13:07:22.687168 3752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:22.687554 3746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:22.687876 3746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:22.689033 3746 hybrid_clock.cc:648] HybridClock initialized: now 1790341642689023 us; error 42 us; skew 500 ppm
Sep 25 13:07:22 dist-test-slave-chkf krb5kdc[3733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341642, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:22.699888 3746 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:07:22.700344 3746 webserver.cc:460] Webserver started at http://127.0.206.126:44293/ using document root <none> and password file <none>
I20260925 13:07:22.700587 3746 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:22.700654 3746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:22.700839 3746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:22.701998 3746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "b5a0af1abb9f4fad8703e37cd66adb15"
format_stamp: "Formatted at 2026-09-25 13:07:22 on dist-test-slave-chkf"
server_key: "0650e83f9c5bbf6c35502c0f6206c68c"
server_key_iv: "4551923ff0ab5cac9f8ed4e0cbe16400"
server_key_version: "encryptionkey@0"
I20260925 13:07:22.702384 3746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "b5a0af1abb9f4fad8703e37cd66adb15"
format_stamp: "Formatted at 2026-09-25 13:07:22 on dist-test-slave-chkf"
server_key: "0650e83f9c5bbf6c35502c0f6206c68c"
server_key_iv: "4551923ff0ab5cac9f8ed4e0cbe16400"
server_key_version: "encryptionkey@0"
I20260925 13:07:22.703768 3746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:07:22.704474 3761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:22.704666 3746 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:22.704761 3746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "b5a0af1abb9f4fad8703e37cd66adb15"
format_stamp: "Formatted at 2026-09-25 13:07:22 on dist-test-slave-chkf"
server_key: "0650e83f9c5bbf6c35502c0f6206c68c"
server_key_iv: "4551923ff0ab5cac9f8ed4e0cbe16400"
server_key_version: "encryptionkey@0"
I20260925 13:07:22.704838 3746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:22.717532 3746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:22.719457 3746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:22.719611 3746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:22.723333 3746 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:37551
I20260925 13:07:22.723379 3813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:37551 every 8 connection(s)
I20260925 13:07:22.723730 3746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:22.724524 3814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:22.724771 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 3746
I20260925 13:07:22.724874 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:07:22.725040 825 external_mini_cluster.cc:1428] Setting key 2c7ac215b67195461f7a0625482ceca6
I20260925 13:07:22.727353 3814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15: Bootstrap starting.
I20260925 13:07:22.727941 3814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:22.728226 3814 log.cc:824] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:22.728855 3814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15: No bootstrap required, opened a new log
I20260925 13:07:22.730482 3814 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a0af1abb9f4fad8703e37cd66adb15" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37551 } }
I20260925 13:07:22.730635 3814 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:22.730682 3814 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a0af1abb9f4fad8703e37cd66adb15, State: Initialized, Role: FOLLOWER
I20260925 13:07:22.730829 3814 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a0af1abb9f4fad8703e37cd66adb15" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37551 } }
I20260925 13:07:22.730902 3814 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:22.730924 3814 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:22.730981 3814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15 [term 0 FOLLOWER]: Advancing to term 1
Sep 25 13:07:22 dist-test-slave-chkf krb5kdc[3733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:22.731520 3814 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a0af1abb9f4fad8703e37cd66adb15" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37551 } }
I20260925 13:07:22.731603 3814 leader_election.cc:302] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5a0af1abb9f4fad8703e37cd66adb15; no voters:
I20260925 13:07:22.731845 3814 leader_election.cc:288] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:07:22.731871 3819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:22.731988 3819 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15 [term 1 LEADER]: Becoming Leader. State: Replica: b5a0af1abb9f4fad8703e37cd66adb15, State: Running, Role: LEADER
I20260925 13:07:22.732074 3819 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a0af1abb9f4fad8703e37cd66adb15" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37551 } }
I20260925 13:07:22.732102 3814 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:22.732497 3819 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5a0af1abb9f4fad8703e37cd66adb15. Latest consensus state: current_term: 1 leader_uuid: "b5a0af1abb9f4fad8703e37cd66adb15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a0af1abb9f4fad8703e37cd66adb15" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37551 } } }
I20260925 13:07:22.732633 3819 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:22.732893 3825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:22.733592 3825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:07:22.733721 3821 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5a0af1abb9f4fad8703e37cd66adb15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a0af1abb9f4fad8703e37cd66adb15" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37551 } } }
I20260925 13:07:22.733872 3821 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:22.735689 3817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:22.725519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41334 (local address 127.0.206.126:37551)
0925 13:07:22.725694 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:22.725701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:22.725725 (+ 24us) server_negotiation.cc:407] Connection header received
0925 13:07:22.726116 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:22.726122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:22.726167 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:22.726296 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:22.727059 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:22.727971 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:22.728789 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:22.729229 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:22.729349 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:22.731746 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:22.731750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:22.731750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:22.731762 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:22.734173 (+ 2411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:22.734542 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:22.734545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:22.734546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:22.734660 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:22.734897 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:22.734897 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:22.734898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:22.735109 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:22.735222 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:22.735357 (+ 135us) server_negotiation.cc:299] Negotiation successful
0925 13:07:22.735458 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260925 13:07:22.736568 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:07:22.739107 3825 catalog_manager.cc:1347] Generated new cluster ID: 28be821feb8b4000a8b19e81760977d2
I20260925 13:07:22.739159 3825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:07:22.754882 3817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:22.751533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41342 (local address 127.0.206.126:37551)
0925 13:07:22.751584 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:22.751586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:22.751728 (+ 142us) server_negotiation.cc:407] Connection header received
0925 13:07:22.751801 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:22.751802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:22.751812 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:22.751885 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:22.752235 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:22.752709 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:22.753428 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:22.753625 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:22.753709 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:22.754381 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:22.754382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:22.754382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:07:22.754587 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:22.754601 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:07:22.754607 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:22.754675 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:22.754773 (+ 98us) server_negotiation.cc:299] Negotiation successful
0925 13:07:22.754795 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 13:07:22.757268 3825 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:07:22.757757 3825 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:22.763230 3825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5a0af1abb9f4fad8703e37cd66adb15: Generated new TSK 0
I20260925 13:07:22.763422 3825 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:07:22.766162 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 3746
2026-09-25T13:07:22Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1318 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:07:22 dist-test-slave-chkf krb5kdc[3847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:07:22 dist-test-slave-chkf krb5kdc[3847](info): set up 2 sockets
Sep 25 13:07:22 dist-test-slave-chkf krb5kdc[3847](info): commencing operation
krb5kdc: starting...
W20260925 13:07:23.790545 825 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:07:23 dist-test-slave-chkf krb5kdc[3847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:07:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:07:23.903473 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:37177
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:37177
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:23.982774 3860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:23.982966 3860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:23.982987 3860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:23.984089 3860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:23.984123 3860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:23.984138 3860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:23.984150 3860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:23.984162 3860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:07:23.985571 3860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:37177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:37177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:23.985761 3860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:23.985935 3860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:23.990962 3865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:23.991016 3868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:23.991025 3860 server_base.cc:1034] running on GCE node
W20260925 13:07:23.990976 3866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:23.991394 3860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:23.991719 3860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:23.992865 3860 hybrid_clock.cc:648] HybridClock initialized: now 1790341643992859 us; error 35 us; skew 500 ppm
Sep 25 13:07:23 dist-test-slave-chkf krb5kdc[3847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341643, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:23.995230 3860 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:07:23.995692 3860 webserver.cc:460] Webserver started at http://127.0.206.126:43691/ using document root <none> and password file <none>
I20260925 13:07:23.995932 3860 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:23.996003 3860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:23.996135 3860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:23.997136 3860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "a33d449afee34a43a6e899c0336dd4fd"
format_stamp: "Formatted at 2026-09-25 13:07:23 on dist-test-slave-chkf"
server_key: "5e7d2695a5907706404af984433aac1f"
server_key_iv: "5e8f3925b0198d6cb015af0c48f42b65"
server_key_version: "encryptionkey@0"
I20260925 13:07:23.997515 3860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "a33d449afee34a43a6e899c0336dd4fd"
format_stamp: "Formatted at 2026-09-25 13:07:23 on dist-test-slave-chkf"
server_key: "5e7d2695a5907706404af984433aac1f"
server_key_iv: "5e8f3925b0198d6cb015af0c48f42b65"
server_key_version: "encryptionkey@0"
I20260925 13:07:23.998832 3860 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:07:23.999526 3875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:23.999691 3860 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:23.999789 3860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "a33d449afee34a43a6e899c0336dd4fd"
format_stamp: "Formatted at 2026-09-25 13:07:23 on dist-test-slave-chkf"
server_key: "5e7d2695a5907706404af984433aac1f"
server_key_iv: "5e8f3925b0198d6cb015af0c48f42b65"
server_key_version: "encryptionkey@0"
I20260925 13:07:23.999868 3860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:24.031332 3860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:24.033210 3860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:24.033378 3860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:24.037498 3860 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:37177
I20260925 13:07:24.037533 3927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:37177 every 8 connection(s)
I20260925 13:07:24.037909 3860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:24.038173 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 3860
I20260925 13:07:24.038281 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:07:24.038489 825 external_mini_cluster.cc:1428] Setting key 74570cbf8fba5d2c6a60d3ae69108635
I20260925 13:07:24.039216 3928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:24.042201 3928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd: Bootstrap starting.
I20260925 13:07:24.042935 3928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:24.043236 3928 log.cc:824] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:24.044296 3928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd: No bootstrap required, opened a new log
Sep 25 13:07:24 dist-test-slave-chkf krb5kdc[3847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:24.046416 3928 raft_consensus.cc:348] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a33d449afee34a43a6e899c0336dd4fd" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37177 } }
I20260925 13:07:24.046617 3928 raft_consensus.cc:374] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:24.046674 3928 raft_consensus.cc:729] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a33d449afee34a43a6e899c0336dd4fd, State: Initialized, Role: FOLLOWER
I20260925 13:07:24.046824 3928 consensus_queue.cc:260] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a33d449afee34a43a6e899c0336dd4fd" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37177 } }
I20260925 13:07:24.046904 3928 raft_consensus.cc:388] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:24.046938 3928 raft_consensus.cc:482] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:24.046980 3928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:24.047677 3928 raft_consensus.cc:504] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a33d449afee34a43a6e899c0336dd4fd" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37177 } }
I20260925 13:07:24.047785 3928 leader_election.cc:302] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a33d449afee34a43a6e899c0336dd4fd; no voters:
I20260925 13:07:24.047955 3928 leader_election.cc:288] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:07:24.047982 3933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:24.048139 3933 raft_consensus.cc:686] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd [term 1 LEADER]: Becoming Leader. State: Replica: a33d449afee34a43a6e899c0336dd4fd, State: Running, Role: LEADER
I20260925 13:07:24.048228 3928 sys_catalog.cc:564] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:24.048210 3933 consensus_queue.cc:237] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a33d449afee34a43a6e899c0336dd4fd" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37177 } }
I20260925 13:07:24.048623 3934 sys_catalog.cc:455] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a33d449afee34a43a6e899c0336dd4fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a33d449afee34a43a6e899c0336dd4fd" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37177 } } }
I20260925 13:07:24.048614 3933 sys_catalog.cc:455] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader a33d449afee34a43a6e899c0336dd4fd. Latest consensus state: current_term: 1 leader_uuid: "a33d449afee34a43a6e899c0336dd4fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a33d449afee34a43a6e899c0336dd4fd" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37177 } } }
I20260925 13:07:24.048705 3934 sys_catalog.cc:458] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:24.048709 3933 sys_catalog.cc:458] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:24.048921 3940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:24.049367 3940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:07:24.050423 3931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:24.039082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45354 (local address 127.0.206.126:37177)
0925 13:07:24.039247 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:24.039256 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:24.039295 (+ 39us) server_negotiation.cc:407] Connection header received
0925 13:07:24.039748 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:24.039755 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:24.039804 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:24.039930 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:24.040706 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:24.041646 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:24.042739 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:24.043380 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:24.043490 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:24.046225 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:24.046229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:24.046229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:24.046241 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:24.048269 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:24.048704 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:24.048708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:24.048709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:24.048825 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:24.049069 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:24.049069 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:24.049070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:24.049406 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:24.050016 (+ 610us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:24.050094 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 13:07:24.050225 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260925 13:07:24.051080 3940 catalog_manager.cc:1347] Generated new cluster ID: 09594fafceef4d21b32aac0005424f5e
I20260925 13:07:24.051136 3940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:07:24.055048 3940 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:07:24.055558 3940 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:24.060576 3940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a33d449afee34a43a6e899c0336dd4fd: Generated new TSK 0
I20260925 13:07:24.060753 3940 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260925 13:07:24.087576 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:37177
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:24.167723 3954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:24.167974 3954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:24.168017 3954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:24.169193 3954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:24.169260 3954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:24.169304 3954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:07:24.170845 3954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:37177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:24.171093 3954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:24.171326 3954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:07:24.176368 3954 server_base.cc:1034] running on GCE node
W20260925 13:07:24.176463 3959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:24.176466 3962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:24.176466 3960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:24.176838 3954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:24.177116 3954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:24.178284 3954 hybrid_clock.cc:648] HybridClock initialized: now 1790341644178268 us; error 44 us; skew 500 ppm
Sep 25 13:07:24 dist-test-slave-chkf krb5kdc[3847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341644, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:24.180732 3954 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:07:24.181226 3954 webserver.cc:460] Webserver started at http://127.0.206.65:41185/ using document root <none> and password file <none>
I20260925 13:07:24.181425 3954 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:24.181468 3954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:24.181553 3954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:24.182518 3954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "42b34d99044a44d2ae245ca4a8ee1e2d"
format_stamp: "Formatted at 2026-09-25 13:07:24 on dist-test-slave-chkf"
server_key: "f5e1a91ffa7b88d66d8ce0d8a695546b"
server_key_iv: "3676212db04f49b45fd59630d306dc13"
server_key_version: "encryptionkey@0"
I20260925 13:07:24.182897 3954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "42b34d99044a44d2ae245ca4a8ee1e2d"
format_stamp: "Formatted at 2026-09-25 13:07:24 on dist-test-slave-chkf"
server_key: "f5e1a91ffa7b88d66d8ce0d8a695546b"
server_key_iv: "3676212db04f49b45fd59630d306dc13"
server_key_version: "encryptionkey@0"
I20260925 13:07:24.184130 3954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:07:24.184789 3969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:24.184970 3954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:07:24.185041 3954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "42b34d99044a44d2ae245ca4a8ee1e2d"
format_stamp: "Formatted at 2026-09-25 13:07:24 on dist-test-slave-chkf"
server_key: "f5e1a91ffa7b88d66d8ce0d8a695546b"
server_key_iv: "3676212db04f49b45fd59630d306dc13"
server_key_version: "encryptionkey@0"
I20260925 13:07:24.185093 3954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:24.213652 3954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:24.215615 3954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:24.215725 3954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:24.215911 3954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:24.216233 3954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:24.216269 3954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:24.216290 3954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:24.216338 3954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:24.222103 3954 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:43679
I20260925 13:07:24.222152 4082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:43679 every 8 connection(s)
I20260925 13:07:24.222467 3954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:07:24.222793 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 3954
I20260925 13:07:24.222916 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:07:24.223130 825 external_mini_cluster.cc:1428] Setting key dfcb8335d051a2fc47a6caf28cbf7e41
Sep 25 13:07:24 dist-test-slave-chkf krb5kdc[3847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341644, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
I20260925 13:07:24.232864 3931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:24.223232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:43325 (local address 127.0.206.126:37177)
0925 13:07:24.223309 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:24.223312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:24.223512 (+ 200us) server_negotiation.cc:407] Connection header received
0925 13:07:24.224110 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:24.224111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:24.224119 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:24.224157 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:24.225501 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:24.226321 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:24.227383 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:24.227757 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:24.227835 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:24.229984 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:24.229986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:24.229986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:24.229995 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:24.231624 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:24.232056 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:24.232057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:24.232058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:24.232118 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:24.232327 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:24.232327 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:24.232327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:24.232426 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:24.232471 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:24.232691 (+ 220us) server_negotiation.cc:299] Negotiation successful
0925 13:07:24.232794 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 13:07:24.233125 4085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:24.223153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:37177 (local address 127.0.206.65:43325)
0925 13:07:24.223370 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:24.223413 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:24.224008 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:24.224228 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:24.224232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:24.225072 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:24.225378 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:24.225385 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:24.226427 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:24.226428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:24.227282 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:24.227284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:24.227845 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:24.227846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:24.227956 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:24.228521 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:24.228529 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:24.229877 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:24.231710 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:24.231716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:24.231717 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:24.231983 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:24.232190 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:24.232191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:24.232191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:24.232286 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:24.232477 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:24.232481 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:24.232589 (+ 108us) client_negotiation.cc:769] Sending connection context
0925 13:07:24.232671 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 13:07:24.232871 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":81,"threads_started":1}
I20260925 13:07:24.233722 4083 heartbeater.cc:347] Connected to a master server at 127.0.206.126:37177
I20260925 13:07:24.233831 4083 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:24.234041 4083 heartbeater.cc:511] Master 127.0.206.126:37177 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
I20260925 13:07:24.234602 3892 ts_manager.cc:194] Registered new tserver with Master: 42b34d99044a44d2ae245ca4a8ee1e2d (127.0.206.65:43679)
I20260925 13:07:24.235245 3892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:43325
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260925 13:07:24.248906 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:37177
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:24.328254 4088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:24.328464 4088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:24.328486 4088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:24.329644 4088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:24.329689 4088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:24.329744 4088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:07:24.331220 4088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:37177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:24.331470 4088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:24.331691 4088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:24.336671 4094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:24.336716 4088 server_base.cc:1034] running on GCE node
W20260925 13:07:24.336696 4093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:24.336696 4096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:24.337056 4088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:24.337304 4088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:24.338475 4088 hybrid_clock.cc:648] HybridClock initialized: now 1790341644338462 us; error 26 us; skew 500 ppm
Sep 25 13:07:24 dist-test-slave-chkf krb5kdc[3847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341644, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:24.340958 4088 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:07:24.341429 4088 webserver.cc:460] Webserver started at http://127.0.206.66:46505/ using document root <none> and password file <none>
I20260925 13:07:24.341670 4088 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:24.341734 4088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:24.341882 4088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:24.343003 4088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "8beae701de6d46229a4863d502ce39de"
format_stamp: "Formatted at 2026-09-25 13:07:24 on dist-test-slave-chkf"
server_key: "dffe2e56ff1840eb6ae58a01174a594f"
server_key_iv: "98bd73d6140c7d3011b2c9a99f7ccbf1"
server_key_version: "encryptionkey@0"
I20260925 13:07:24.343367 4088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "8beae701de6d46229a4863d502ce39de"
format_stamp: "Formatted at 2026-09-25 13:07:24 on dist-test-slave-chkf"
server_key: "dffe2e56ff1840eb6ae58a01174a594f"
server_key_iv: "98bd73d6140c7d3011b2c9a99f7ccbf1"
server_key_version: "encryptionkey@0"
I20260925 13:07:24.344617 4088 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:24.345326 4103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:24.345496 4088 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:24.345592 4088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "8beae701de6d46229a4863d502ce39de"
format_stamp: "Formatted at 2026-09-25 13:07:24 on dist-test-slave-chkf"
server_key: "dffe2e56ff1840eb6ae58a01174a594f"
server_key_iv: "98bd73d6140c7d3011b2c9a99f7ccbf1"
server_key_version: "encryptionkey@0"
I20260925 13:07:24.345674 4088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:24.359364 4088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:24.361161 4088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:24.361325 4088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:24.361557 4088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:24.361868 4088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:24.361922 4088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:24.361963 4088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:24.361991 4088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:24.367654 4088 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:37357
I20260925 13:07:24.367693 4216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:37357 every 8 connection(s)
I20260925 13:07:24.368073 4088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:07:24.373643 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 4088
I20260925 13:07:24.373754 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:07:24.373974 825 external_mini_cluster.cc:1428] Setting key f5d4047cd5326ac140cfa02b3d607365
Sep 25 13:07:24 dist-test-slave-chkf krb5kdc[3847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341644, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:24.377988 4219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:24.368819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:37177 (local address 127.0.206.66:57701)
0925 13:07:24.369002 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:24.369037 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:24.369538 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:24.369792 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:24.369796 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:24.370381 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:24.370569 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:24.370574 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:24.371285 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:24.371286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:24.372080 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:24.372083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:24.372475 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:24.372476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:24.372621 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:24.373052 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:24.373059 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:24.374773 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:24.376571 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:24.376577 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:24.376579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:24.376897 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:24.377083 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:24.377084 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:24.377084 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:24.377222 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:24.377474 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:24.377478 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:24.377629 (+ 151us) client_negotiation.cc:769] Sending connection context
0925 13:07:24.377717 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 13:07:24.377832 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":96,"threads_started":1}
I20260925 13:07:24.377980 3931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:24.368856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:57701 (local address 127.0.206.126:37177)
0925 13:07:24.368902 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:24.368904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:24.369120 (+ 216us) server_negotiation.cc:407] Connection header received
0925 13:07:24.369642 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:24.369643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:24.369654 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:24.369720 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:24.370666 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:24.371191 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:24.372181 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:24.372394 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:24.372471 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:24.374879 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:24.374880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:24.374881 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:24.374889 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:24.376480 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:24.376970 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:24.376971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:24.376971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:24.377020 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:24.377291 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:24.377291 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:24.377292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:24.377384 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:24.377476 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:24.377762 (+ 286us) server_negotiation.cc:299] Negotiation successful
0925 13:07:24.377848 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 13:07:24.378623 4217 heartbeater.cc:347] Connected to a master server at 127.0.206.126:37177
I20260925 13:07:24.378750 4217 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:24.378931 4217 heartbeater.cc:511] Master 127.0.206.126:37177 requested a full tablet report, sending...
I20260925 13:07:24.379336 3892 ts_manager.cc:194] Registered new tserver with Master: 8beae701de6d46229a4863d502ce39de (127.0.206.66:37357)
I20260925 13:07:24.379700 3892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:57701
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260925 13:07:24.399642 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:37177
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:24.479261 4222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:24.479485 4222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:24.479519 4222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:24.480664 4222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:24.480729 4222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:24.480774 4222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:07:24.482218 4222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:37177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:24.482479 4222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:24.482724 4222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:24.487624 4228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:24.487640 4230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:24.487665 4227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:24.487830 4222 server_base.cc:1034] running on GCE node
I20260925 13:07:24.488188 4222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:24.488374 4222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:24.489518 4222 hybrid_clock.cc:648] HybridClock initialized: now 1790341644489506 us; error 27 us; skew 500 ppm
Sep 25 13:07:24 dist-test-slave-chkf krb5kdc[3847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341644, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:24.492039 4222 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:07:24.492509 4222 webserver.cc:460] Webserver started at http://127.0.206.67:41477/ using document root <none> and password file <none>
I20260925 13:07:24.492723 4222 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:24.492795 4222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:24.492936 4222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:24.493947 4222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "95fa13420c7340c5a9c7f35366bc404f"
format_stamp: "Formatted at 2026-09-25 13:07:24 on dist-test-slave-chkf"
server_key: "9631f1d49623ac9a6eb8800315057e45"
server_key_iv: "fd51592b066d8c532b30030d92cf0830"
server_key_version: "encryptionkey@0"
I20260925 13:07:24.494307 4222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "95fa13420c7340c5a9c7f35366bc404f"
format_stamp: "Formatted at 2026-09-25 13:07:24 on dist-test-slave-chkf"
server_key: "9631f1d49623ac9a6eb8800315057e45"
server_key_iv: "fd51592b066d8c532b30030d92cf0830"
server_key_version: "encryptionkey@0"
I20260925 13:07:24.495661 4222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:07:24.496382 4237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:24.496546 4222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:24.496644 4222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "95fa13420c7340c5a9c7f35366bc404f"
format_stamp: "Formatted at 2026-09-25 13:07:24 on dist-test-slave-chkf"
server_key: "9631f1d49623ac9a6eb8800315057e45"
server_key_iv: "fd51592b066d8c532b30030d92cf0830"
server_key_version: "encryptionkey@0"
I20260925 13:07:24.496716 4222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:24.524437 4222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:24.526933 4222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:24.527092 4222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:24.527335 4222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:24.527639 4222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:24.527690 4222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:24.527725 4222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:24.527771 4222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:24.533524 4222 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:35573
I20260925 13:07:24.533568 4350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:35573 every 8 connection(s)
I20260925 13:07:24.533882 4222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:07:24.534426 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 4222
I20260925 13:07:24.534536 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:07:24.534808 825 external_mini_cluster.cc:1428] Setting key bc1bdbfebc0986b04492aa293f2f546f
Sep 25 13:07:24 dist-test-slave-chkf krb5kdc[3847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341644, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:24.542919 3931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:24.534595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:42803 (local address 127.0.206.126:37177)
0925 13:07:24.534636 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:24.534638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:24.534932 (+ 294us) server_negotiation.cc:407] Connection header received
0925 13:07:24.535594 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:24.535595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:24.535609 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:24.535663 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:24.536600 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:24.537094 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:24.538106 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:24.538301 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:24.538391 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:24.540426 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:24.540428 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:24.540428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:24.540440 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:24.541554 (+ 1114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:24.541961 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:24.541963 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:24.541963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:24.542021 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:24.542257 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:24.542258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:24.542258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:24.542383 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:24.542475 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:24.542750 (+ 275us) server_negotiation.cc:299] Negotiation successful
0925 13:07:24.542827 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:07:24.543118 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:24.534591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:37177 (local address 127.0.206.67:42803)
0925 13:07:24.534775 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:24.534824 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:24.535478 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:24.535738 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:24.535742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:24.536316 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:24.536506 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:24.536511 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:24.537192 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:24.537193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:24.538002 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:24.538003 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:24.538399 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:24.538400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:24.538533 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:24.539013 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:24.539021 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:24.540313 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:24.541638 (+ 1325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:24.541644 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:24.541666 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:24.541885 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:24.542087 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:24.542088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:24.542089 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:24.542190 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:24.542469 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:24.542475 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:24.542615 (+ 140us) client_negotiation.cc:769] Sending connection context
0925 13:07:24.542740 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 13:07:24.542887 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":117,"threads_started":1}
I20260925 13:07:24.543581 4351 heartbeater.cc:347] Connected to a master server at 127.0.206.126:37177
I20260925 13:07:24.543677 4351 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:24.543849 4351 heartbeater.cc:511] Master 127.0.206.126:37177 requested a full tablet report, sending...
I20260925 13:07:24.544175 3892 ts_manager.cc:194] Registered new tserver with Master: 95fa13420c7340c5a9c7f35366bc404f (127.0.206.67:35573)
I20260925 13:07:24.544565 3892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:42803
I20260925 13:07:24.546553 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 13:07:24 dist-test-slave-chkf krb5kdc[3847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341644, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 25 13:07:24 dist-test-slave-chkf krb5kdc[3847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341644, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:24.568583 3931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:24.561847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45364 (local address 127.0.206.126:37177)
0925 13:07:24.561913 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:24.561916 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:24.562060 (+ 144us) server_negotiation.cc:407] Connection header received
0925 13:07:24.562106 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:24.562107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:24.562116 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:24.562166 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:24.562700 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:24.563238 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:24.563936 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:24.564105 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:24.564152 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:24.566271 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:24.566272 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:24.566273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:24.566279 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:24.567558 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:24.567869 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:24.567870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:24.567870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:24.567920 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:24.568080 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:24.568081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:24.568081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:24.568210 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:24.568283 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:24.568400 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 13:07:24.568464 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260925 13:07:24.568964 3892 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45364
I20260925 13:07:24.569402 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 3954
I20260925 13:07:24.574215 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 4088
I20260925 13:07:24.578481 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 4222
I20260925 13:07:24.582957 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 3860
2026-09-25T13:07:24Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1819 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:07:24 dist-test-slave-chkf krb5kdc[4363](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:07:24 dist-test-slave-chkf krb5kdc[4363](info): set up 2 sockets
Sep 25 13:07:24 dist-test-slave-chkf krb5kdc[4363](info): commencing operation
krb5kdc: starting...
W20260925 13:07:25.614549 825 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:07:25 dist-test-slave-chkf krb5kdc[4363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:07:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:07:25.725739 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:33785
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:33785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:25.804203 4376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:25.804441 4376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:25.804481 4376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:25.805639 4376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:25.805698 4376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:25.805722 4376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:25.805750 4376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:25.805775 4376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:07:25.807222 4376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:33785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:33785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:25.807500 4376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:25.807716 4376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:25.812433 4381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:25.812467 4382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:25.812485 4376 server_base.cc:1034] running on GCE node
W20260925 13:07:25.812443 4384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:25.812919 4376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:25.813197 4376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:25.814339 4376 hybrid_clock.cc:648] HybridClock initialized: now 1790341645814332 us; error 28 us; skew 500 ppm
Sep 25 13:07:25 dist-test-slave-chkf krb5kdc[4363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:25.816656 4376 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:07:25.817059 4376 webserver.cc:460] Webserver started at http://127.0.206.126:36377/ using document root <none> and password file <none>
I20260925 13:07:25.817273 4376 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:25.817335 4376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:25.817483 4376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:25.818538 4376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "186234f0eab14eb0aa7a02930831dc16"
format_stamp: "Formatted at 2026-09-25 13:07:25 on dist-test-slave-chkf"
server_key: "464aaf0cd54d60f2f9018ff6c6ad1772"
server_key_iv: "40a9c9001be57215284cb7c06bc73d6d"
server_key_version: "encryptionkey@0"
I20260925 13:07:25.818913 4376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "186234f0eab14eb0aa7a02930831dc16"
format_stamp: "Formatted at 2026-09-25 13:07:25 on dist-test-slave-chkf"
server_key: "464aaf0cd54d60f2f9018ff6c6ad1772"
server_key_iv: "40a9c9001be57215284cb7c06bc73d6d"
server_key_version: "encryptionkey@0"
I20260925 13:07:25.820241 4376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:07:25.820922 4391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:25.821080 4376 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:07:25.821179 4376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "186234f0eab14eb0aa7a02930831dc16"
format_stamp: "Formatted at 2026-09-25 13:07:25 on dist-test-slave-chkf"
server_key: "464aaf0cd54d60f2f9018ff6c6ad1772"
server_key_iv: "40a9c9001be57215284cb7c06bc73d6d"
server_key_version: "encryptionkey@0"
I20260925 13:07:25.821249 4376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:25.845283 4376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:25.847226 4376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:25.847388 4376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:25.851249 4376 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:33785
I20260925 13:07:25.851293 4443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:33785 every 8 connection(s)
I20260925 13:07:25.851663 4376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:25.852490 4444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:25.854779 4444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16: Bootstrap starting.
I20260925 13:07:25.855350 4444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:25.855639 4444 log.cc:824] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:25.856288 4444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16: No bootstrap required, opened a new log
I20260925 13:07:25.857852 4444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186234f0eab14eb0aa7a02930831dc16" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33785 } }
I20260925 13:07:25.858009 4444 raft_consensus.cc:374] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:25.858063 4444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186234f0eab14eb0aa7a02930831dc16, State: Initialized, Role: FOLLOWER
I20260925 13:07:25.858224 4444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186234f0eab14eb0aa7a02930831dc16" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33785 } }
I20260925 13:07:25.858296 4444 raft_consensus.cc:388] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:25.858335 4444 raft_consensus.cc:482] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:25.858385 4444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:25.858917 4444 raft_consensus.cc:504] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186234f0eab14eb0aa7a02930831dc16" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33785 } }
I20260925 13:07:25.859057 4444 leader_election.cc:302] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 186234f0eab14eb0aa7a02930831dc16; no voters:
I20260925 13:07:25.859258 4444 leader_election.cc:288] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:07:25.859295 4447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:25.859519 4444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:25.859629 4447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16 [term 1 LEADER]: Becoming Leader. State: Replica: 186234f0eab14eb0aa7a02930831dc16, State: Running, Role: LEADER
I20260925 13:07:25.859717 4447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186234f0eab14eb0aa7a02930831dc16" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33785 } }
I20260925 13:07:25.859970 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 4376
I20260925 13:07:25.860077 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:07:25.860270 825 external_mini_cluster.cc:1428] Setting key 6c608526ff674ad8d32ba5dcec873d58
I20260925 13:07:25.860966 4449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 186234f0eab14eb0aa7a02930831dc16. Latest consensus state: current_term: 1 leader_uuid: "186234f0eab14eb0aa7a02930831dc16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186234f0eab14eb0aa7a02930831dc16" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33785 } } }
I20260925 13:07:25.861049 4449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:25.861369 4448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "186234f0eab14eb0aa7a02930831dc16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186234f0eab14eb0aa7a02930831dc16" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33785 } } }
I20260925 13:07:25.861614 4448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16 [sys.catalog]: This master's current role is: LEADER
W20260925 13:07:25.861717 4462 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:07:25.861758 4462 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:07:25.861814 4463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:25.862000 4463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:07:25.863411 4463 catalog_manager.cc:1347] Generated new cluster ID: b827931e18d544118214ef5b2d9fe342
I20260925 13:07:25.863462 4463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 13:07:25 dist-test-slave-chkf krb5kdc[4363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:25.871256 4463 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:07:25.871833 4463 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:25.872853 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:25.861308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56282 (local address 127.0.206.126:33785)
0925 13:07:25.861471 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:25.861483 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:25.861516 (+ 33us) server_negotiation.cc:407] Connection header received
0925 13:07:25.861637 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:25.861645 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:25.861714 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:25.861894 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:25.862541 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:25.863965 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:25.865029 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:25.865674 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:25.865792 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:25.868643 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:25.868647 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:25.868647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:25.868660 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:25.870695 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:25.871185 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:25.871189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:25.871190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:25.871302 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:25.871549 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:25.871550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:25.871550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:25.871842 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:25.872404 (+ 562us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:25.872479 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 13:07:25.872630 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260925 13:07:25.876622 4463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 186234f0eab14eb0aa7a02930831dc16: Generated new TSK 0
I20260925 13:07:25.876821 4463 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260925 13:07:25.906064 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33785
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:25.986017 4470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:25.986210 4470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:25.986233 4470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:25.987417 4470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:25.987458 4470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:25.987505 4470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:07:25.988953 4470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:25.989180 4470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:25.989394 4470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:25.994261 4475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:25.994274 4476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:25.994279 4478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:25.994468 4470 server_base.cc:1034] running on GCE node
I20260925 13:07:25.994735 4470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:25.994998 4470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:25.996166 4470 hybrid_clock.cc:648] HybridClock initialized: now 1790341645996150 us; error 29 us; skew 500 ppm
Sep 25 13:07:25 dist-test-slave-chkf krb5kdc[4363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:25.998517 4470 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:07:25.998992 4470 webserver.cc:460] Webserver started at http://127.0.206.65:45189/ using document root <none> and password file <none>
I20260925 13:07:25.999212 4470 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:25.999274 4470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:25.999420 4470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:26.000437 4470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "f57ed1bf5b95486491f16fc4be896038"
format_stamp: "Formatted at 2026-09-25 13:07:25 on dist-test-slave-chkf"
server_key: "92021f899c1d322ae72379d3f4dc1a42"
server_key_iv: "055067941ce49108f1edb9e2020b4ca7"
server_key_version: "encryptionkey@0"
I20260925 13:07:26.000806 4470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "f57ed1bf5b95486491f16fc4be896038"
format_stamp: "Formatted at 2026-09-25 13:07:25 on dist-test-slave-chkf"
server_key: "92021f899c1d322ae72379d3f4dc1a42"
server_key_iv: "055067941ce49108f1edb9e2020b4ca7"
server_key_version: "encryptionkey@0"
I20260925 13:07:26.002068 4470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:07:26.002784 4485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:26.002952 4470 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:26.003029 4470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "f57ed1bf5b95486491f16fc4be896038"
format_stamp: "Formatted at 2026-09-25 13:07:25 on dist-test-slave-chkf"
server_key: "92021f899c1d322ae72379d3f4dc1a42"
server_key_iv: "055067941ce49108f1edb9e2020b4ca7"
server_key_version: "encryptionkey@0"
I20260925 13:07:26.003080 4470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:26.010766 4470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:26.012398 4470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:26.012501 4470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:26.012686 4470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:26.012917 4470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:26.012945 4470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:26.012965 4470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:26.012977 4470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:26.018496 4470 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:36917
I20260925 13:07:26.018537 4598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:36917 every 8 connection(s)
I20260925 13:07:26.018886 4470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:07:26.020607 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 4470
I20260925 13:07:26.020725 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:07:26.020915 825 external_mini_cluster.cc:1428] Setting key b82835a3b6371800cd0953f9def63068
Sep 25 13:07:26 dist-test-slave-chkf krb5kdc[4363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
I20260925 13:07:26.029136 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:26.019621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:38737 (local address 127.0.206.126:33785)
0925 13:07:26.019684 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:26.019686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:26.019885 (+ 199us) server_negotiation.cc:407] Connection header received
0925 13:07:26.020831 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:26.020834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:26.020846 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:26.020908 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:26.021723 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:26.022217 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:26.023696 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:26.023969 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:26.024024 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:26.026351 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:26.026353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:26.026353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:26.026362 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:26.027811 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:26.028206 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:26.028207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:26.028208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:26.028263 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:26.028509 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:26.028510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:26.028510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:26.028624 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:26.028697 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:26.028958 (+ 261us) server_negotiation.cc:299] Negotiation successful
0925 13:07:26.029046 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 13:07:26.029311 4601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:26.019613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33785 (local address 127.0.206.65:38737)
0925 13:07:26.019760 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:26.019800 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:26.020686 (+ 886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:26.020997 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:26.021001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:26.021361 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:26.021630 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:26.021636 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:26.022308 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:26.022309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:26.023604 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:26.023608 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:26.024044 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:26.024045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:26.024189 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:26.024667 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:26.024677 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:26.026257 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:26.027891 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:26.027898 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:26.027901 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:26.028122 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:26.028325 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:26.028325 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:26.028326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:26.028443 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:26.028687 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:26.028693 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:26.028829 (+ 136us) client_negotiation.cc:769] Sending connection context
0925 13:07:26.028933 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 13:07:26.029082 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260925 13:07:26.029753 4599 heartbeater.cc:347] Connected to a master server at 127.0.206.126:33785
I20260925 13:07:26.029892 4599 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:26.030081 4599 heartbeater.cc:511] Master 127.0.206.126:33785 requested a full tablet report, sending...
I20260925 13:07:26.030571 4408 ts_manager.cc:194] Registered new tserver with Master: f57ed1bf5b95486491f16fc4be896038 (127.0.206.65:36917)
I20260925 13:07:26.031185 4408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:38737
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260925 13:07:26.044974 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33785
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:26.124769 4604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:26.124980 4604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:26.125005 4604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:26.126160 4604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:26.126204 4604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:26.126230 4604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:07:26.127702 4604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:26.127959 4604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:26.128187 4604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:26.133169 4609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:26.133255 4604 server_base.cc:1034] running on GCE node
W20260925 13:07:26.133180 4612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:26.133183 4610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:26.133596 4604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:26.133838 4604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:26.134984 4604 hybrid_clock.cc:648] HybridClock initialized: now 1790341646134966 us; error 36 us; skew 500 ppm
Sep 25 13:07:26 dist-test-slave-chkf krb5kdc[4363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341646, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:26.137450 4604 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:07:26.137905 4604 webserver.cc:460] Webserver started at http://127.0.206.66:46523/ using document root <none> and password file <none>
I20260925 13:07:26.138145 4604 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:26.138190 4604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:26.138345 4604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:26.139418 4604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "0d336f5faa9e4cd58452ceb60a9a32b7"
format_stamp: "Formatted at 2026-09-25 13:07:26 on dist-test-slave-chkf"
server_key: "9e761dce337b45c06a9eb5d85ffb9cad"
server_key_iv: "73427aaa1a0f67658fb9f94b961bee0a"
server_key_version: "encryptionkey@0"
I20260925 13:07:26.139760 4604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "0d336f5faa9e4cd58452ceb60a9a32b7"
format_stamp: "Formatted at 2026-09-25 13:07:26 on dist-test-slave-chkf"
server_key: "9e761dce337b45c06a9eb5d85ffb9cad"
server_key_iv: "73427aaa1a0f67658fb9f94b961bee0a"
server_key_version: "encryptionkey@0"
I20260925 13:07:26.141057 4604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:07:26.141716 4619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:26.141876 4604 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:26.141978 4604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "0d336f5faa9e4cd58452ceb60a9a32b7"
format_stamp: "Formatted at 2026-09-25 13:07:26 on dist-test-slave-chkf"
server_key: "9e761dce337b45c06a9eb5d85ffb9cad"
server_key_iv: "73427aaa1a0f67658fb9f94b961bee0a"
server_key_version: "encryptionkey@0"
I20260925 13:07:26.142045 4604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:26.166549 4604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:26.168900 4604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:26.169059 4604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:26.169286 4604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:26.169641 4604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:26.169689 4604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:26.169730 4604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:26.169761 4604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:26.175333 4604 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:39413
I20260925 13:07:26.175382 4732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:39413 every 8 connection(s)
I20260925 13:07:26.175680 4604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:07:26.179701 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 4604
I20260925 13:07:26.179826 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:07:26.179994 825 external_mini_cluster.cc:1428] Setting key b45c37e419516fea40b49ff275d1b687
Sep 25 13:07:26 dist-test-slave-chkf krb5kdc[4363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341646, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:26.185226 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:26.176379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:41693 (local address 127.0.206.126:33785)
0925 13:07:26.176443 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:26.176445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:26.176648 (+ 203us) server_negotiation.cc:407] Connection header received
0925 13:07:26.177550 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:26.177551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:26.177561 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:26.177618 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:26.178177 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:26.178791 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:26.179798 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:26.180072 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:26.180145 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:26.182269 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:26.182270 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:26.182271 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:26.182279 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:26.183878 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:26.184243 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:26.184245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:26.184245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:26.184311 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:26.184555 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:26.184556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:26.184557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:26.184702 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:26.184770 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:26.185009 (+ 239us) server_negotiation.cc:299] Negotiation successful
0925 13:07:26.185096 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 13:07:26.185237 4735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:26.176346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33785 (local address 127.0.206.66:41693)
0925 13:07:26.176528 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:26.176570 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:26.177467 (+ 897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:26.177673 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:26.177676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:26.177924 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:26.178112 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:26.178115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:26.178875 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:26.178876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:26.179701 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:26.179703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:26.180154 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:26.180155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:26.180290 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:26.180844 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:26.180852 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:26.182175 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:26.183954 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:26.183959 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:26.183961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:26.184168 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:26.184386 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:26.184387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:26.184387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:26.184492 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:26.184773 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:26.184776 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:26.184885 (+ 109us) client_negotiation.cc:769] Sending connection context
0925 13:07:26.184965 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 13:07:26.185083 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":110,"threads_started":1}
I20260925 13:07:26.185786 4733 heartbeater.cc:347] Connected to a master server at 127.0.206.126:33785
I20260925 13:07:26.185875 4733 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:26.186064 4733 heartbeater.cc:511] Master 127.0.206.126:33785 requested a full tablet report, sending...
I20260925 13:07:26.186407 4408 ts_manager.cc:194] Registered new tserver with Master: 0d336f5faa9e4cd58452ceb60a9a32b7 (127.0.206.66:39413)
I20260925 13:07:26.186838 4408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:41693
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260925 13:07:26.203732 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33785
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:26.283246 4738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:26.283470 4738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:26.283520 4738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:26.284652 4738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:26.284718 4738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:26.284761 4738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:07:26.286217 4738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:26.286476 4738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:26.286697 4738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:26.291457 4743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:26.291471 4744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:26.291539 4738 server_base.cc:1034] running on GCE node
W20260925 13:07:26.291460 4746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:26.291917 4738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:26.292105 4738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:26.293267 4738 hybrid_clock.cc:648] HybridClock initialized: now 1790341646293254 us; error 31 us; skew 500 ppm
Sep 25 13:07:26 dist-test-slave-chkf krb5kdc[4363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341646, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:26.295624 4738 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:07:26.296020 4738 webserver.cc:460] Webserver started at http://127.0.206.67:36197/ using document root <none> and password file <none>
I20260925 13:07:26.296245 4738 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:26.296308 4738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:26.296448 4738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:26.297446 4738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "50c85f3c97fc46aa86487816cc6a71ec"
format_stamp: "Formatted at 2026-09-25 13:07:26 on dist-test-slave-chkf"
server_key: "d8e9ac53548befa5b7675dc9dacfea8c"
server_key_iv: "bb5475959c288aeff186049d718065e9"
server_key_version: "encryptionkey@0"
I20260925 13:07:26.297784 4738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "50c85f3c97fc46aa86487816cc6a71ec"
format_stamp: "Formatted at 2026-09-25 13:07:26 on dist-test-slave-chkf"
server_key: "d8e9ac53548befa5b7675dc9dacfea8c"
server_key_iv: "bb5475959c288aeff186049d718065e9"
server_key_version: "encryptionkey@0"
I20260925 13:07:26.299100 4738 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 13:07:26.299742 4753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:26.299894 4738 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:26.299984 4738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "50c85f3c97fc46aa86487816cc6a71ec"
format_stamp: "Formatted at 2026-09-25 13:07:26 on dist-test-slave-chkf"
server_key: "d8e9ac53548befa5b7675dc9dacfea8c"
server_key_iv: "bb5475959c288aeff186049d718065e9"
server_key_version: "encryptionkey@0"
I20260925 13:07:26.300053 4738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:26.315166 4738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:26.316974 4738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:26.317127 4738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:26.317371 4738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:26.317667 4738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:26.317715 4738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:26.317747 4738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:26.317775 4738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:26.323412 4738 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:35861
I20260925 13:07:26.323457 4866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:35861 every 8 connection(s)
I20260925 13:07:26.323959 4738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:07:26.328723 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 4738
I20260925 13:07:26.328814 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:07:26.328982 825 external_mini_cluster.cc:1428] Setting key f2c386797ea1c58f9d4d77e3f0e5c0a6
Sep 25 13:07:26 dist-test-slave-chkf krb5kdc[4363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341646, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:26.332811 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:26.324660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:45151 (local address 127.0.206.126:33785)
0925 13:07:26.324730 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:26.324732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:26.324917 (+ 185us) server_negotiation.cc:407] Connection header received
0925 13:07:26.325789 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:26.325790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:26.325800 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:26.325840 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:26.326384 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:26.326951 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:26.327893 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:26.328068 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:26.328117 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:26.330380 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:26.330381 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:26.330381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:26.330387 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:26.331773 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:26.332113 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:26.332114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:26.332114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:26.332160 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:26.332356 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:26.332357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:26.332357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:26.332453 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:26.332501 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:26.332697 (+ 196us) server_negotiation.cc:299] Negotiation successful
0925 13:07:26.332751 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 13:07:26.333016 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:26.324641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33785 (local address 127.0.206.67:45151)
0925 13:07:26.324798 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:26.324838 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:26.325706 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:26.325894 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:26.325897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:26.326136 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:26.326321 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:26.326325 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:26.327017 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:26.327018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:26.327825 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:26.327827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:26.328132 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:26.328132 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:26.328224 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:26.328684 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:26.328692 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:26.330311 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:26.331843 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:26.331848 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:26.331850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:26.332054 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:26.332211 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:26.332212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:26.332212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:26.332308 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:26.332503 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:26.332506 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:26.332609 (+ 103us) client_negotiation.cc:769] Sending connection context
0925 13:07:26.332687 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 13:07:26.332793 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260925 13:07:26.333441 4867 heartbeater.cc:347] Connected to a master server at 127.0.206.126:33785
I20260925 13:07:26.333559 4867 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:26.333721 4867 heartbeater.cc:511] Master 127.0.206.126:33785 requested a full tablet report, sending...
I20260925 13:07:26.334046 4408 ts_manager.cc:194] Registered new tserver with Master: 50c85f3c97fc46aa86487816cc6a71ec (127.0.206.67:35861)
I20260925 13:07:26.334503 4408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:45151
I20260925 13:07:26.340615 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 13:07:26 dist-test-slave-chkf krb5kdc[4363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 13:07:26 dist-test-slave-chkf krb5kdc[4363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260925 13:07:26.363062 4874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:26.355192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35544 (local address 127.0.206.65:36917)
0925 13:07:26.355353 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:26.355359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:26.355372 (+ 13us) server_negotiation.cc:407] Connection header received
0925 13:07:26.355404 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:26.355407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:26.355424 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:26.355507 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:26.356066 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:26.356777 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:26.357512 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:26.357835 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:26.357912 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:26.360272 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:26.360281 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:26.360281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:26.360293 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:26.361894 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:26.362240 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:26.362249 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:26.362249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:26.362300 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:26.362489 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:26.362490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:26.362491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:26.362620 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:26.362820 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:26.362864 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 13:07:26.362944 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
Sep 25 13:07:26 dist-test-slave-chkf krb5kdc[4363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:26.370594 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:26.364164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56290 (local address 127.0.206.126:33785)
0925 13:07:26.364205 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:26.364207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:26.364295 (+ 88us) server_negotiation.cc:407] Connection header received
0925 13:07:26.364354 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:26.364355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:26.364363 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:26.364400 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:26.364796 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:26.365235 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:26.365825 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:26.365986 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:26.366032 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:26.368227 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:26.368228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:26.368229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:26.368235 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:26.369516 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:26.369893 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:26.369895 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:26.369895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:26.369959 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:26.370118 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:26.370119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:26.370119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:26.370245 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:26.370317 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:26.370396 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 13:07:26.370471 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 13:07:26.370888 4408 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56290
I20260925 13:07:26.371233 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 4470
I20260925 13:07:26.375978 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 4604
I20260925 13:07:26.380626 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 4738
I20260925 13:07:26.384595 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 4376
2026-09-25T13:07:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1801 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:07:26 dist-test-slave-chkf krb5kdc[4879](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:07:26 dist-test-slave-chkf krb5kdc[4879](info): set up 2 sockets
Sep 25 13:07:26 dist-test-slave-chkf krb5kdc[4879](info): commencing operation
krb5kdc: starting...
W20260925 13:07:27.414665 825 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:07:27 dist-test-slave-chkf krb5kdc[4879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:07:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:07:27.531646 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:45703
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:45703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:27.609282 4892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:27.609469 4892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:27.609490 4892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:27.610582 4892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:27.610620 4892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:27.610633 4892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:27.610668 4892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:27.610693 4892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:07:27.612201 4892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:45703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:45703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:27.612434 4892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:27.612643 4892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:27.617092 4898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:27.617121 4892 server_base.cc:1034] running on GCE node
W20260925 13:07:27.617092 4897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:27.617153 4900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:27.617580 4892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:27.617842 4892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:27.618992 4892 hybrid_clock.cc:648] HybridClock initialized: now 1790341647618979 us; error 29 us; skew 500 ppm
Sep 25 13:07:27 dist-test-slave-chkf krb5kdc[4879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341647, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:27.621210 4892 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:07:27.621524 4892 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:27.621589 4892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:27.621723 4892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:27.622687 4892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "9669b1b78a764e6ba0cdbcd1e02495ff"
format_stamp: "Formatted at 2026-09-25 13:07:27 on dist-test-slave-chkf"
server_key: "70c0b1587942038b88177069ba942e20"
server_key_iv: "f2a7543a09b62c1b66dd2b8b75b591a2"
server_key_version: "encryptionkey@0"
I20260925 13:07:27.623086 4892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "9669b1b78a764e6ba0cdbcd1e02495ff"
format_stamp: "Formatted at 2026-09-25 13:07:27 on dist-test-slave-chkf"
server_key: "70c0b1587942038b88177069ba942e20"
server_key_iv: "f2a7543a09b62c1b66dd2b8b75b591a2"
server_key_version: "encryptionkey@0"
I20260925 13:07:27.624359 4892 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:27.625239 4906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:27.625424 4892 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:27.625524 4892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "9669b1b78a764e6ba0cdbcd1e02495ff"
format_stamp: "Formatted at 2026-09-25 13:07:27 on dist-test-slave-chkf"
server_key: "70c0b1587942038b88177069ba942e20"
server_key_iv: "f2a7543a09b62c1b66dd2b8b75b591a2"
server_key_version: "encryptionkey@0"
I20260925 13:07:27.625597 4892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:27.645736 4892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:27.647948 4892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:27.648053 4892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:27.651573 4892 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:45703
I20260925 13:07:27.651633 4958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:45703 every 8 connection(s)
I20260925 13:07:27.651899 4892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:27.652868 4959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:27.655083 4959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff: Bootstrap starting.
I20260925 13:07:27.655632 4959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:27.655864 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 4892
I20260925 13:07:27.655974 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:07:27.655985 4959 log.cc:824] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:27.656162 825 external_mini_cluster.cc:1428] Setting key 5aea9b72536829a1a23d5a4390be040a
I20260925 13:07:27.656854 4959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff: No bootstrap required, opened a new log
I20260925 13:07:27.659381 4959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9669b1b78a764e6ba0cdbcd1e02495ff" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45703 } }
I20260925 13:07:27.659541 4959 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:27.659577 4959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9669b1b78a764e6ba0cdbcd1e02495ff, State: Initialized, Role: FOLLOWER
I20260925 13:07:27.659715 4959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9669b1b78a764e6ba0cdbcd1e02495ff" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45703 } }
I20260925 13:07:27.659790 4959 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:27.659812 4959 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:27.659880 4959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:27.660420 4959 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9669b1b78a764e6ba0cdbcd1e02495ff" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45703 } }
I20260925 13:07:27.660521 4959 leader_election.cc:302] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9669b1b78a764e6ba0cdbcd1e02495ff; no voters:
I20260925 13:07:27.660667 4959 leader_election.cc:288] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:07:27.660722 4964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:27.660869 4964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff [term 1 LEADER]: Becoming Leader. State: Replica: 9669b1b78a764e6ba0cdbcd1e02495ff, State: Running, Role: LEADER
I20260925 13:07:27.660899 4959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:27.660964 4964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9669b1b78a764e6ba0cdbcd1e02495ff" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45703 } }
I20260925 13:07:27.661250 4964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9669b1b78a764e6ba0cdbcd1e02495ff. Latest consensus state: current_term: 1 leader_uuid: "9669b1b78a764e6ba0cdbcd1e02495ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9669b1b78a764e6ba0cdbcd1e02495ff" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45703 } } }
I20260925 13:07:27.661322 4964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:27.661544 4966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9669b1b78a764e6ba0cdbcd1e02495ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9669b1b78a764e6ba0cdbcd1e02495ff" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45703 } } }
I20260925 13:07:27.661993 4973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:27.662235 4966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:27.662740 4973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 13:07:27 dist-test-slave-chkf krb5kdc[4879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:27.664981 4973 catalog_manager.cc:1347] Generated new cluster ID: 6f54e036dfba41c1b23554bdb2e0c831
I20260925 13:07:27.665036 4973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:07:27.673202 4963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:27.656614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40638 (local address 127.0.206.126:45703)
0925 13:07:27.656794 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:27.656802 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:27.656841 (+ 39us) server_negotiation.cc:407] Connection header received
0925 13:07:27.657333 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:27.657341 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:27.657403 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:27.657556 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:27.658094 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:27.659561 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:27.660393 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:27.661026 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:27.661211 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:27.664032 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:27.664036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:27.664037 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:27.664047 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:27.671641 (+ 7594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:27.672037 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:27.672040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:27.672041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:27.672145 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:27.672332 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:27.672332 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:27.672333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:27.672615 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:27.672733 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:27.672869 (+ 136us) server_negotiation.cc:299] Negotiation successful
0925 13:07:27.672987 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
I20260925 13:07:27.688699 4973 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:07:27.689471 4973 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:27.699163 4973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff: Generated new TSK 0
I20260925 13:07:27.699375 4973 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260925 13:07:27.729131 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45703
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:27.824626 4985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:27.824807 4985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:27.824828 4985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:27.825868 4985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:27.825901 4985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:27.825932 4985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:07:27.827418 4985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:27.827595 4985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:27.827778 4985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:27.832645 4990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:27.832674 4991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:27.832679 4993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:27.832695 4985 server_base.cc:1034] running on GCE node
I20260925 13:07:27.832984 4985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:27.833166 4985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:27.834301 4985 hybrid_clock.cc:648] HybridClock initialized: now 1790341647834288 us; error 32 us; skew 500 ppm
Sep 25 13:07:27 dist-test-slave-chkf krb5kdc[4879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341647, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:27.841389 4985 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:07:27.841742 4985 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:27.841789 4985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:27.841876 4985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:27.842926 4985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "89495dc9f23a4bd69c02f15673019889"
format_stamp: "Formatted at 2026-09-25 13:07:27 on dist-test-slave-chkf"
server_key: "5416238c8d58fe588f6c5ebf4ae7f258"
server_key_iv: "035218d480fff83c5adb873b0e76348e"
server_key_version: "encryptionkey@0"
I20260925 13:07:27.843214 4985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "89495dc9f23a4bd69c02f15673019889"
format_stamp: "Formatted at 2026-09-25 13:07:27 on dist-test-slave-chkf"
server_key: "5416238c8d58fe588f6c5ebf4ae7f258"
server_key_iv: "035218d480fff83c5adb873b0e76348e"
server_key_version: "encryptionkey@0"
I20260925 13:07:27.844415 4985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 13:07:27.845053 4999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:27.845222 4985 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:27.845291 4985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "89495dc9f23a4bd69c02f15673019889"
format_stamp: "Formatted at 2026-09-25 13:07:27 on dist-test-slave-chkf"
server_key: "5416238c8d58fe588f6c5ebf4ae7f258"
server_key_iv: "035218d480fff83c5adb873b0e76348e"
server_key_version: "encryptionkey@0"
I20260925 13:07:27.845341 4985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:27.865389 4985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:27.867259 4985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:27.867376 4985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:27.867551 4985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:27.867866 4985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:27.867899 4985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:27.867956 4985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:27.867981 4985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:27.873550 4985 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:39257
I20260925 13:07:27.873586 5112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:39257 every 8 connection(s)
I20260925 13:07:27.873832 4985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:07:27.874282 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 4985
I20260925 13:07:27.874393 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:07:27.874593 825 external_mini_cluster.cc:1428] Setting key 7e3c09a6a772d472a546749560cdd872
Sep 25 13:07:27 dist-test-slave-chkf krb5kdc[4879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341647, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
I20260925 13:07:27.883324 4963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:27.874492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:50305 (local address 127.0.206.126:45703)
0925 13:07:27.874794 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:27.874813 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:27.874827 (+ 14us) server_negotiation.cc:407] Connection header received
0925 13:07:27.875266 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:27.875267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:27.875278 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:27.875319 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:27.876576 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:27.877349 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:27.878295 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:27.878509 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:27.878589 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:27.880621 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:27.880622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:27.880623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:27.880630 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:27.881979 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:27.882383 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:27.882385 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:27.882386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:27.882441 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:27.882683 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:27.882684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:27.882684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:27.882808 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:27.882887 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:27.883140 (+ 253us) server_negotiation.cc:299] Negotiation successful
0925 13:07:27.883245 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273}
I20260925 13:07:27.883500 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:27.874494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45703 (local address 127.0.206.65:50305)
0925 13:07:27.874661 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:27.874720 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:27.875185 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:27.875414 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:27.875418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:27.875806 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:27.876486 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:27.876493 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:27.877453 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:27.877454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:27.878204 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:27.878205 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:27.878599 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:27.878600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:27.878770 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:27.879257 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:27.879266 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:27.880535 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:27.882056 (+ 1521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:27.882062 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:27.882065 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:27.882300 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:27.882501 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:27.882502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:27.882502 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:27.882615 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:27.882870 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:27.882875 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:27.883005 (+ 130us) client_negotiation.cc:769] Sending connection context
0925 13:07:27.883101 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 13:07:27.883279 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260925 13:07:27.883931 5113 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45703
I20260925 13:07:27.884029 5113 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:27.884135 5113 heartbeater.cc:511] Master 127.0.206.126:45703 requested a full tablet report, sending...
I20260925 13:07:27.884588 4923 ts_manager.cc:194] Registered new tserver with Master: 89495dc9f23a4bd69c02f15673019889 (127.0.206.65:39257)
I20260925 13:07:27.885176 4923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:50305
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260925 13:07:27.899631 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45703
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:27.979483 5118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:27.979738 5118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:27.979775 5118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:27.982127 5118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:27.982187 5118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:27.982228 5118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:07:27.985133 5118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:27.985416 5118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:27.985673 5118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:27.992197 5126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:27.992197 5124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:27.992218 5123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:27.993132 5118 server_base.cc:1034] running on GCE node
I20260925 13:07:27.993340 5118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:27.993523 5118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:27.994673 5118 hybrid_clock.cc:648] HybridClock initialized: now 1790341647994659 us; error 31 us; skew 500 ppm
Sep 25 13:07:27 dist-test-slave-chkf krb5kdc[4879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341647, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:27.997798 5118 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:07:27.998185 5118 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:27.998256 5118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:27.998404 5118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:27.999939 5118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "933b71c1cffc4359858f612c0e055596"
format_stamp: "Formatted at 2026-09-25 13:07:27 on dist-test-slave-chkf"
server_key: "d78a487cde77db10f13186008bc4a13e"
server_key_iv: "d9dfea64e8dfb47af8577cc419f1b839"
server_key_version: "encryptionkey@0"
I20260925 13:07:28.000388 5118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "933b71c1cffc4359858f612c0e055596"
format_stamp: "Formatted at 2026-09-25 13:07:27 on dist-test-slave-chkf"
server_key: "d78a487cde77db10f13186008bc4a13e"
server_key_iv: "d9dfea64e8dfb47af8577cc419f1b839"
server_key_version: "encryptionkey@0"
I20260925 13:07:28.002161 5118 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260925 13:07:28.002943 5132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:28.003161 5118 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:28.003261 5118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "933b71c1cffc4359858f612c0e055596"
format_stamp: "Formatted at 2026-09-25 13:07:27 on dist-test-slave-chkf"
server_key: "d78a487cde77db10f13186008bc4a13e"
server_key_iv: "d9dfea64e8dfb47af8577cc419f1b839"
server_key_version: "encryptionkey@0"
I20260925 13:07:28.003341 5118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:28.024240 5118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:28.026782 5118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:28.026906 5118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:28.027092 5118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:28.027441 5118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:28.027504 5118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:28.027546 5118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:28.027573 5118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:28.033378 5118 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:38029
I20260925 13:07:28.033412 5245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:38029 every 8 connection(s)
I20260925 13:07:28.033679 5118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:07:28.033986 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 5118
I20260925 13:07:28.034090 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:07:28.034278 825 external_mini_cluster.cc:1428] Setting key fda06256f45df13adb1bac2aa1ee8b14
Sep 25 13:07:28 dist-test-slave-chkf krb5kdc[4879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341647, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
I20260925 13:07:28.043537 4963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.034449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:55555 (local address 127.0.206.126:45703)
0925 13:07:28.034498 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:28.034500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:28.034732 (+ 232us) server_negotiation.cc:407] Connection header received
0925 13:07:28.035266 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:28.035267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:28.035278 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:28.035346 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:28.036324 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.036785 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.037814 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.038137 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.038219 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:28.040720 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:28.040721 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:28.040722 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:28.040730 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:28.042157 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:28.042555 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:28.042556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:28.042556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:28.042613 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:28.042896 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:28.042897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:28.042897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:28.043017 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:28.043104 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:28.043410 (+ 306us) server_negotiation.cc:299] Negotiation successful
0925 13:07:28.043474 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 13:07:28.043756 5248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.034422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45703 (local address 127.0.206.66:55555)
0925 13:07:28.034579 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:28.034623 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:28.035176 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:28.035398 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:28.035402 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:28.035635 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:28.036231 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:28.036239 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.036864 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.036865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:28.037731 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:28.037734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.038221 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.038221 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:28.038314 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:28.039065 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:28.039076 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:28.040628 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:28.042246 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:28.042252 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:28.042253 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:28.042483 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:28.042680 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:28.042681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:28.042681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:28.042812 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:28.043093 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:28.043099 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:28.043257 (+ 158us) client_negotiation.cc:769] Sending connection context
0925 13:07:28.043370 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 13:07:28.043514 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260925 13:07:28.044245 5246 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45703
I20260925 13:07:28.044391 5246 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:28.044570 5246 heartbeater.cc:511] Master 127.0.206.126:45703 requested a full tablet report, sending...
I20260925 13:07:28.044852 4923 ts_manager.cc:194] Registered new tserver with Master: 933b71c1cffc4359858f612c0e055596 (127.0.206.66:38029)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
I20260925 13:07:28.045313 4923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:55555
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260925 13:07:28.060688 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45703
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:28.139065 5251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:28.139261 5251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:28.139283 5251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:28.140357 5251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:28.140393 5251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:28.140417 5251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:07:28.141872 5251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:28.142082 5251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:28.142261 5251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:28.147014 5257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:28.147011 5256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:28.147014 5259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:28.147261 5251 server_base.cc:1034] running on GCE node
I20260925 13:07:28.147490 5251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:28.147706 5251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:28.148885 5251 hybrid_clock.cc:648] HybridClock initialized: now 1790341648148872 us; error 29 us; skew 500 ppm
Sep 25 13:07:28 dist-test-slave-chkf krb5kdc[4879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341648, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:28.151409 5251 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:07:28.151782 5251 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:28.151829 5251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:28.151983 5251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:28.152994 5251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "c21f5bb4959d48cdb50d0282f2aa1dde"
format_stamp: "Formatted at 2026-09-25 13:07:28 on dist-test-slave-chkf"
server_key: "281a98618a56cbcb66cc569e4de2d857"
server_key_iv: "658eb7b91dea3426483646ce89bb274e"
server_key_version: "encryptionkey@0"
I20260925 13:07:28.153330 5251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "c21f5bb4959d48cdb50d0282f2aa1dde"
format_stamp: "Formatted at 2026-09-25 13:07:28 on dist-test-slave-chkf"
server_key: "281a98618a56cbcb66cc569e4de2d857"
server_key_iv: "658eb7b91dea3426483646ce89bb274e"
server_key_version: "encryptionkey@0"
I20260925 13:07:28.154654 5251 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 13:07:28.155369 5265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:28.155534 5251 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:07:28.155617 5251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "c21f5bb4959d48cdb50d0282f2aa1dde"
format_stamp: "Formatted at 2026-09-25 13:07:28 on dist-test-slave-chkf"
server_key: "281a98618a56cbcb66cc569e4de2d857"
server_key_iv: "658eb7b91dea3426483646ce89bb274e"
server_key_version: "encryptionkey@0"
I20260925 13:07:28.155686 5251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:28.171600 5251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:28.173727 5251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:28.173893 5251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:28.174089 5251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:28.174368 5251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:28.174417 5251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:28.174456 5251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:28.174481 5251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:28.180114 5251 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:39063
I20260925 13:07:28.180148 5378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:39063 every 8 connection(s)
I20260925 13:07:28.180392 5251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:07:28.185141 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 5251
I20260925 13:07:28.185223 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:07:28.185386 825 external_mini_cluster.cc:1428] Setting key 0230b24ba07ce1e14ce67cb467c8f27d
Sep 25 13:07:28 dist-test-slave-chkf krb5kdc[4879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341648, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:28.188648 4963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.181022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:42913 (local address 127.0.206.126:45703)
0925 13:07:28.181068 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:28.181071 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:28.181274 (+ 203us) server_negotiation.cc:407] Connection header received
0925 13:07:28.181716 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:28.181718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:28.181727 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:28.181789 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:28.182623 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.183280 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.184216 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.184384 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.184430 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:28.186422 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:28.186423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:28.186423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:28.186430 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:28.187575 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:28.187934 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:28.187935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:28.187935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:28.187983 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:28.188193 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:28.188194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:28.188194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:28.188286 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:28.188334 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:28.188537 (+ 203us) server_negotiation.cc:299] Negotiation successful
0925 13:07:28.188589 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 13:07:28.188877 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.181016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45703 (local address 127.0.206.67:42913)
0925 13:07:28.181161 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:28.181194 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:28.181628 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:28.181840 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:28.181843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:28.182087 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:28.182553 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:28.182557 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.183364 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.183365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:28.184154 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:28.184156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.184446 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.184446 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:28.184537 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:28.184900 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:28.184913 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:28.186362 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:28.187635 (+ 1273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:28.187640 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:28.187641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:28.187867 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:28.188051 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:28.188051 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:28.188052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:28.188143 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:28.188334 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:28.188338 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:28.188443 (+ 105us) client_negotiation.cc:769] Sending connection context
0925 13:07:28.188525 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 13:07:28.188633 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260925 13:07:28.189366 5379 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45703
I20260925 13:07:28.189461 5379 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:28.189610 5379 heartbeater.cc:511] Master 127.0.206.126:45703 requested a full tablet report, sending...
I20260925 13:07:28.189915 4923 ts_manager.cc:194] Registered new tserver with Master: c21f5bb4959d48cdb50d0282f2aa1dde (127.0.206.67:39063)
I20260925 13:07:28.190227 4923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:42913
I20260925 13:07:28.196856 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:07:28.203399 4963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.198026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40648 (local address 127.0.206.126:45703)
0925 13:07:28.198063 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:28.198065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:28.198138 (+ 73us) server_negotiation.cc:407] Connection header received
0925 13:07:28.198185 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:28.198186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:28.198193 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:28.198241 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:28.198755 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.199298 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.200032 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.200201 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.200252 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:28.201347 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:28.201348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:28.201348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:28.201353 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:28.202474 (+ 1121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:28.202789 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:28.202789 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:28.202790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:28.202836 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:28.202983 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:28.202984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:28.202984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:28.203117 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:28.203188 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:28.203261 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 13:07:28.203316 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:07:28.204780 4923 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:07:28.206274 4923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:07:28.212946 5392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.208626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50086 (local address 127.0.206.67:39063)
0925 13:07:28.208787 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:28.208792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:28.209356 (+ 564us) server_negotiation.cc:407] Connection header received
0925 13:07:28.209395 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:28.209397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:28.209411 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:28.209483 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:28.209608 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.210668 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.211802 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.212537 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.212619 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:28.212680 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:28.212821 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 13:07:28.212859 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260925 13:07:28.212955 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.208537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:39063 (local address 127.0.0.1:50086)
0925 13:07:28.208803 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:28.208846 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:28.208934 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:28.209479 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:28.209480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:28.209482 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:28.209531 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:28.209533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.210790 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.210791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:28.211720 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:28.211722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.212629 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.212630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:28.212710 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:28.212726 (+ 16us) client_negotiation.cc:769] Sending connection context
0925 13:07:28.212802 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 13:07:28.212872 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":54,"thread_start_us":50,"threads_started":1}
I20260925 13:07:28.213182 5393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.208629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57144 (local address 127.0.206.65:39257)
0925 13:07:28.209086 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:28.209090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:28.209098 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:28.209159 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:28.209161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:28.209176 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:28.209249 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:28.209649 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.210317 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.211876 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.212574 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.212637 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:28.212682 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:28.213054 (+ 372us) server_negotiation.cc:299] Negotiation successful
0925 13:07:28.213101 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":84,"threads_started":1}
I20260925 13:07:28.212955 5394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.208537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:39257 (local address 127.0.0.1:57144)
0925 13:07:28.208879 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:28.208888 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:28.208934 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:28.209555 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:28.209556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:28.209557 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:28.209592 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:28.209593 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.210441 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.210442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:28.211789 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:28.211792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.212640 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.212641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:28.212709 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:28.212726 (+ 17us) client_negotiation.cc:769] Sending connection context
0925 13:07:28.212802 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 13:07:28.212872 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":69,"thread_start_us":72,"threads_started":1}
I20260925 13:07:28.213723 5313 tablet_service.cc:1467] Processing CreateTablet for tablet 8c13cc0f103e4cc6950e1a17189502ec (DEFAULT_TABLE table=test-table [id=1dcdddb44c394a30aafb1bad1372a26f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:28.213941 5390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.208786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38029 (local address 127.0.0.1:54670)
0925 13:07:28.208890 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:28.208901 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:28.208973 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:28.209217 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:28.209226 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:28.209228 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:28.209309 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:28.209311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.210700 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.210701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:28.211805 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:28.211807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.213707 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.213708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:28.213779 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:28.213783 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:07:28.213835 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 13:07:28.213874 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":3,"thread_start_us":57,"threads_started":1}
I20260925 13:07:28.213979 5395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.208870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54670 (local address 127.0.206.66:38029)
0925 13:07:28.209058 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:28.209062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:28.209071 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:28.209110 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:28.209112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:28.209123 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:28.209207 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:28.209431 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.210602 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.212716 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.213553 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.213622 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:28.213674 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:28.213864 (+ 190us) server_negotiation.cc:299] Negotiation successful
0925 13:07:28.213905 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":91,"threads_started":1}
I20260925 13:07:28.214097 5313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c13cc0f103e4cc6950e1a17189502ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:28.214138 5047 tablet_service.cc:1467] Processing CreateTablet for tablet 8c13cc0f103e4cc6950e1a17189502ec (DEFAULT_TABLE table=test-table [id=1dcdddb44c394a30aafb1bad1372a26f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:28.214429 5047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c13cc0f103e4cc6950e1a17189502ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:28.214748 5180 tablet_service.cc:1467] Processing CreateTablet for tablet 8c13cc0f103e4cc6950e1a17189502ec (DEFAULT_TABLE table=test-table [id=1dcdddb44c394a30aafb1bad1372a26f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:28.215046 5180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c13cc0f103e4cc6950e1a17189502ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:28.216920 5396 tablet_bootstrap.cc:492] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde: Bootstrap starting.
I20260925 13:07:28.217460 5396 tablet_bootstrap.cc:654] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:28.218035 5396 log.cc:824] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:28.218137 5398 tablet_bootstrap.cc:492] T 8c13cc0f103e4cc6950e1a17189502ec P 933b71c1cffc4359858f612c0e055596: Bootstrap starting.
I20260925 13:07:28.218472 5397 tablet_bootstrap.cc:492] T 8c13cc0f103e4cc6950e1a17189502ec P 89495dc9f23a4bd69c02f15673019889: Bootstrap starting.
I20260925 13:07:28.218819 5396 tablet_bootstrap.cc:492] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde: No bootstrap required, opened a new log
I20260925 13:07:28.218851 5398 tablet_bootstrap.cc:654] T 8c13cc0f103e4cc6950e1a17189502ec P 933b71c1cffc4359858f612c0e055596: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:28.218891 5396 ts_tablet_manager.cc:1397] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:07:28.219108 5397 tablet_bootstrap.cc:654] T 8c13cc0f103e4cc6950e1a17189502ec P 89495dc9f23a4bd69c02f15673019889: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:28.219451 5398 log.cc:824] T 8c13cc0f103e4cc6950e1a17189502ec P 933b71c1cffc4359858f612c0e055596: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:28.219732 5397 log.cc:824] T 8c13cc0f103e4cc6950e1a17189502ec P 89495dc9f23a4bd69c02f15673019889: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:28.220136 5398 tablet_bootstrap.cc:492] T 8c13cc0f103e4cc6950e1a17189502ec P 933b71c1cffc4359858f612c0e055596: No bootstrap required, opened a new log
I20260925 13:07:28.220217 5398 ts_tablet_manager.cc:1397] T 8c13cc0f103e4cc6950e1a17189502ec P 933b71c1cffc4359858f612c0e055596: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:07:28.220470 5397 tablet_bootstrap.cc:492] T 8c13cc0f103e4cc6950e1a17189502ec P 89495dc9f23a4bd69c02f15673019889: No bootstrap required, opened a new log
I20260925 13:07:28.220533 5397 ts_tablet_manager.cc:1397] T 8c13cc0f103e4cc6950e1a17189502ec P 89495dc9f23a4bd69c02f15673019889: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:07:28.220830 5396 raft_consensus.cc:348] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21f5bb4959d48cdb50d0282f2aa1dde" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39063 } } peers { permanent_uuid: "89495dc9f23a4bd69c02f15673019889" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39257 } } peers { permanent_uuid: "933b71c1cffc4359858f612c0e055596" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38029 } }
I20260925 13:07:28.220973 5396 raft_consensus.cc:374] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:28.221014 5396 raft_consensus.cc:729] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c21f5bb4959d48cdb50d0282f2aa1dde, State: Initialized, Role: FOLLOWER
I20260925 13:07:28.221120 5396 consensus_queue.cc:260] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21f5bb4959d48cdb50d0282f2aa1dde" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39063 } } peers { permanent_uuid: "89495dc9f23a4bd69c02f15673019889" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39257 } } peers { permanent_uuid: "933b71c1cffc4359858f612c0e055596" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38029 } }
I20260925 13:07:28.221385 5396 ts_tablet_manager.cc:1428] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:07:28.221511 5379 heartbeater.cc:503] Master 127.0.206.126:45703 was elected leader, sending a full tablet report...
I20260925 13:07:28.222155 5398 raft_consensus.cc:348] T 8c13cc0f103e4cc6950e1a17189502ec P 933b71c1cffc4359858f612c0e055596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21f5bb4959d48cdb50d0282f2aa1dde" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39063 } } peers { permanent_uuid: "89495dc9f23a4bd69c02f15673019889" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39257 } } peers { permanent_uuid: "933b71c1cffc4359858f612c0e055596" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38029 } }
I20260925 13:07:28.222321 5398 raft_consensus.cc:374] T 8c13cc0f103e4cc6950e1a17189502ec P 933b71c1cffc4359858f612c0e055596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:28.222360 5398 raft_consensus.cc:729] T 8c13cc0f103e4cc6950e1a17189502ec P 933b71c1cffc4359858f612c0e055596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 933b71c1cffc4359858f612c0e055596, State: Initialized, Role: FOLLOWER
I20260925 13:07:28.222466 5398 consensus_queue.cc:260] T 8c13cc0f103e4cc6950e1a17189502ec P 933b71c1cffc4359858f612c0e055596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21f5bb4959d48cdb50d0282f2aa1dde" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39063 } } peers { permanent_uuid: "89495dc9f23a4bd69c02f15673019889" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39257 } } peers { permanent_uuid: "933b71c1cffc4359858f612c0e055596" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38029 } }
I20260925 13:07:28.222535 5397 raft_consensus.cc:348] T 8c13cc0f103e4cc6950e1a17189502ec P 89495dc9f23a4bd69c02f15673019889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21f5bb4959d48cdb50d0282f2aa1dde" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39063 } } peers { permanent_uuid: "89495dc9f23a4bd69c02f15673019889" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39257 } } peers { permanent_uuid: "933b71c1cffc4359858f612c0e055596" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38029 } }
I20260925 13:07:28.222684 5397 raft_consensus.cc:374] T 8c13cc0f103e4cc6950e1a17189502ec P 89495dc9f23a4bd69c02f15673019889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:28.222756 5397 raft_consensus.cc:729] T 8c13cc0f103e4cc6950e1a17189502ec P 89495dc9f23a4bd69c02f15673019889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89495dc9f23a4bd69c02f15673019889, State: Initialized, Role: FOLLOWER
I20260925 13:07:28.222776 5398 ts_tablet_manager.cc:1428] T 8c13cc0f103e4cc6950e1a17189502ec P 933b71c1cffc4359858f612c0e055596: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 13:07:28.222797 5246 heartbeater.cc:503] Master 127.0.206.126:45703 was elected leader, sending a full tablet report...
I20260925 13:07:28.222901 5397 consensus_queue.cc:260] T 8c13cc0f103e4cc6950e1a17189502ec P 89495dc9f23a4bd69c02f15673019889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21f5bb4959d48cdb50d0282f2aa1dde" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39063 } } peers { permanent_uuid: "89495dc9f23a4bd69c02f15673019889" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39257 } } peers { permanent_uuid: "933b71c1cffc4359858f612c0e055596" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38029 } }
I20260925 13:07:28.223177 5397 ts_tablet_manager.cc:1428] T 8c13cc0f103e4cc6950e1a17189502ec P 89495dc9f23a4bd69c02f15673019889: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 13:07:28.223300 5113 heartbeater.cc:503] Master 127.0.206.126:45703 was elected leader, sending a full tablet report...
W20260925 13:07:28.284198 5247 tablet.cc:2369] T 8c13cc0f103e4cc6950e1a17189502ec P 933b71c1cffc4359858f612c0e055596: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:07:28.319682 5402 raft_consensus.cc:482] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:28.319821 5402 raft_consensus.cc:504] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21f5bb4959d48cdb50d0282f2aa1dde" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39063 } } peers { permanent_uuid: "89495dc9f23a4bd69c02f15673019889" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39257 } } peers { permanent_uuid: "933b71c1cffc4359858f612c0e055596" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38029 } }
I20260925 13:07:28.320191 5402 leader_election.cc:288] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89495dc9f23a4bd69c02f15673019889 (127.0.206.65:39257), 933b71c1cffc4359858f612c0e055596 (127.0.206.66:38029)
I20260925 13:07:28.323522 5395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.320704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:46061 (local address 127.0.206.66:38029)
0925 13:07:28.320775 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:28.320777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:28.320788 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:07:28.320845 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:28.320846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:28.320856 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:28.320926 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:28.321076 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.321533 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.322596 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.323180 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.323260 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:28.323270 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:28.323440 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 13:07:28.323470 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 13:07:28.323602 5405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.320534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38029 (local address 127.0.206.67:46061)
0925 13:07:28.320717 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:28.320741 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:28.320790 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:28.320941 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:28.320942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:28.320944 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:28.321011 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:28.321012 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.321612 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.321613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:28.322512 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:28.322515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.323266 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.323267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:28.323353 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:28.323358 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 13:07:28.323418 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 13:07:28.323466 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":19,"thread_start_us":109,"threads_started":1}
I20260925 13:07:28.323971 5200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c13cc0f103e4cc6950e1a17189502ec" candidate_uuid: "c21f5bb4959d48cdb50d0282f2aa1dde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933b71c1cffc4359858f612c0e055596" is_pre_election: true
I20260925 13:07:28.324150 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.320400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:39257 (local address 127.0.206.67:49875)
0925 13:07:28.320459 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:28.320472 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:28.320518 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:28.320865 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:28.320866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:28.320868 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:28.320949 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:28.320951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.321614 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.321615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:28.322495 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:28.322497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.323926 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.323927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:28.323984 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:28.323986 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:07:28.324033 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 13:07:28.324058 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":66}
I20260925 13:07:28.324147 5200 raft_consensus.cc:2454] T 8c13cc0f103e4cc6950e1a17189502ec P 933b71c1cffc4359858f612c0e055596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c21f5bb4959d48cdb50d0282f2aa1dde in term 0.
I20260925 13:07:28.324164 5393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.320613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:49875 (local address 127.0.206.65:39257)
0925 13:07:28.320688 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:28.320690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:28.320702 (+ 12us) server_negotiation.cc:407] Connection header received
0925 13:07:28.320733 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:28.320734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:28.320744 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:28.320844 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:28.321022 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.321532 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.322630 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.323201 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.323265 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:28.323271 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:28.324060 (+ 789us) server_negotiation.cc:299] Negotiation successful
0925 13:07:28.324104 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 13:07:28.324404 5266 leader_election.cc:302] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 933b71c1cffc4359858f612c0e055596, c21f5bb4959d48cdb50d0282f2aa1dde; no voters:
I20260925 13:07:28.324491 5067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c13cc0f103e4cc6950e1a17189502ec" candidate_uuid: "c21f5bb4959d48cdb50d0282f2aa1dde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89495dc9f23a4bd69c02f15673019889" is_pre_election: true
I20260925 13:07:28.324546 5402 raft_consensus.cc:2781] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:07:28.324602 5402 raft_consensus.cc:482] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:07:28.324631 5402 raft_consensus.cc:3037] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:28.324667 5067 raft_consensus.cc:2454] T 8c13cc0f103e4cc6950e1a17189502ec P 89495dc9f23a4bd69c02f15673019889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c21f5bb4959d48cdb50d0282f2aa1dde in term 0.
I20260925 13:07:28.373121 5402 raft_consensus.cc:504] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21f5bb4959d48cdb50d0282f2aa1dde" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39063 } } peers { permanent_uuid: "89495dc9f23a4bd69c02f15673019889" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39257 } } peers { permanent_uuid: "933b71c1cffc4359858f612c0e055596" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38029 } }
I20260925 13:07:28.373346 5402 leader_election.cc:288] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [CANDIDATE]: Term 1 election: Requested vote from peers 89495dc9f23a4bd69c02f15673019889 (127.0.206.65:39257), 933b71c1cffc4359858f612c0e055596 (127.0.206.66:38029)
I20260925 13:07:28.373579 5067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c13cc0f103e4cc6950e1a17189502ec" candidate_uuid: "c21f5bb4959d48cdb50d0282f2aa1dde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89495dc9f23a4bd69c02f15673019889"
I20260925 13:07:28.373638 5200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c13cc0f103e4cc6950e1a17189502ec" candidate_uuid: "c21f5bb4959d48cdb50d0282f2aa1dde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933b71c1cffc4359858f612c0e055596"
I20260925 13:07:28.373715 5200 raft_consensus.cc:3037] T 8c13cc0f103e4cc6950e1a17189502ec P 933b71c1cffc4359858f612c0e055596 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:28.373703 5067 raft_consensus.cc:3037] T 8c13cc0f103e4cc6950e1a17189502ec P 89495dc9f23a4bd69c02f15673019889 [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:07:28.374469 5114 tablet.cc:2369] T 8c13cc0f103e4cc6950e1a17189502ec P 89495dc9f23a4bd69c02f15673019889: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:07:28.374627 5067 raft_consensus.cc:2454] T 8c13cc0f103e4cc6950e1a17189502ec P 89495dc9f23a4bd69c02f15673019889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c21f5bb4959d48cdb50d0282f2aa1dde in term 1.
I20260925 13:07:28.374627 5200 raft_consensus.cc:2454] T 8c13cc0f103e4cc6950e1a17189502ec P 933b71c1cffc4359858f612c0e055596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c21f5bb4959d48cdb50d0282f2aa1dde in term 1.
I20260925 13:07:28.374833 5266 leader_election.cc:302] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89495dc9f23a4bd69c02f15673019889, c21f5bb4959d48cdb50d0282f2aa1dde; no voters:
I20260925 13:07:28.374966 5402 raft_consensus.cc:2781] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:28.375164 5402 raft_consensus.cc:686] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [term 1 LEADER]: Becoming Leader. State: Replica: c21f5bb4959d48cdb50d0282f2aa1dde, State: Running, Role: LEADER
I20260925 13:07:28.375277 5402 consensus_queue.cc:237] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21f5bb4959d48cdb50d0282f2aa1dde" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39063 } } peers { permanent_uuid: "89495dc9f23a4bd69c02f15673019889" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39257 } } peers { permanent_uuid: "933b71c1cffc4359858f612c0e055596" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38029 } }
I20260925 13:07:28.376161 4923 catalog_manager.cc:5495] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde reported cstate change: term changed from 0 to 1, leader changed from <none> to c21f5bb4959d48cdb50d0282f2aa1dde (127.0.206.67). New cstate: current_term: 1 leader_uuid: "c21f5bb4959d48cdb50d0282f2aa1dde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c21f5bb4959d48cdb50d0282f2aa1dde" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 39063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89495dc9f23a4bd69c02f15673019889" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "933b71c1cffc4359858f612c0e055596" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38029 } health_report { overall_health: UNKNOWN } } }
I20260925 13:07:28.392048 5392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.389471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50100 (local address 127.0.206.67:39063)
0925 13:07:28.389511 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:28.389513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:28.389521 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:28.389591 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:28.389592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:28.389611 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:28.389703 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:28.389840 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.390355 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.391247 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:28.391479 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:28.391556 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:28.391701 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:28.391809 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:28.391899 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:28.391924 (+ 25us) server_negotiation.cc:299] Negotiation successful
0925 13:07:28.391947 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 13:07:28.393702 5200 raft_consensus.cc:1261] T 8c13cc0f103e4cc6950e1a17189502ec P 933b71c1cffc4359858f612c0e055596 [term 1 FOLLOWER]: Refusing update from remote peer c21f5bb4959d48cdb50d0282f2aa1dde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:28.393726 5067 raft_consensus.cc:1261] T 8c13cc0f103e4cc6950e1a17189502ec P 89495dc9f23a4bd69c02f15673019889 [term 1 FOLLOWER]: Refusing update from remote peer c21f5bb4959d48cdb50d0282f2aa1dde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:28.394054 5402 consensus_queue.cc:1035] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [LEADER]: Connected to new peer: Peer: permanent_uuid: "933b71c1cffc4359858f612c0e055596" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:28.394182 5406 consensus_queue.cc:1035] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [LEADER]: Connected to new peer: Peer: permanent_uuid: "89495dc9f23a4bd69c02f15673019889" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:28.396595 5410 mvcc.cc:205] Tried to move back new op lower bound from 7333239391818182656 to 7333239391745875968. Current Snapshot: MvccSnapshot[applied={T|T < 7333239391818182656}]
I20260925 13:07:28.396936 5411 mvcc.cc:205] Tried to move back new op lower bound from 7333239391818182656 to 7333239391745875968. Current Snapshot: MvccSnapshot[applied={T|T < 7333239391818182656 or (T in {7333239391818182656})}]
I20260925 13:07:28.398764 4923 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:28.398854 4923 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:28.399542 4923 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9669b1b78a764e6ba0cdbcd1e02495ff: Sending DeleteTablet for 3 replicas of tablet 8c13cc0f103e4cc6950e1a17189502ec
I20260925 13:07:28.399921 5047 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c13cc0f103e4cc6950e1a17189502ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:28 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:57144
I20260925 13:07:28.399914 5180 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c13cc0f103e4cc6950e1a17189502ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:28 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:54670
I20260925 13:07:28.400162 5313 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c13cc0f103e4cc6950e1a17189502ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:28 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:50086
I20260925 13:07:28.400343 5421 tablet_replica.cc:330] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde: stopping tablet replica
I20260925 13:07:28.400418 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 4985
I20260925 13:07:28.400434 5421 raft_consensus.cc:2229] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:07:28.400908 5421 raft_consensus.cc:2258] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:28.401352 5421 ts_tablet_manager.cc:1905] T 8c13cc0f103e4cc6950e1a17189502ec P c21f5bb4959d48cdb50d0282f2aa1dde: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:07:28.406482 4907 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:39257 (error 108)
I20260925 13:07:28.406939 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 5118
I20260925 13:07:28.407051 5390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.406928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:39257 (local address 127.0.0.1:57154)
0925 13:07:28.406970 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:28.407011 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260925 13:07:28.407699 4907 catalog_manager.cc:4584] TS 89495dc9f23a4bd69c02f15673019889 (127.0.206.65:39257): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8c13cc0f103e4cc6950e1a17189502ec: Network error: Client connection negotiation failed: client connection to 127.0.206.65:39257: connect: Connection refused (error 111)
I20260925 13:07:28.412015 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 5251
I20260925 13:07:28.412047 5390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.411952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38029 (local address 127.0.0.1:54672)
0925 13:07:28.411994 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:28.412021 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.66:38029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260925 13:07:28.456477 5390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.456301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38029 (local address 127.0.0.1:54676)
0925 13:07:28.456386 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:28.456438 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.66:38029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260925 13:07:28.456842 5390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.456763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38029 (local address 127.0.0.1:54692)
0925 13:07:28.456800 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:28.456825 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.66:38029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260925 13:07:28.460230 5390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.460144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:39257 (local address 127.0.0.1:57164)
0925 13:07:28.460185 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:28.460211 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 13:07:28.460515 5390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.460445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:39257 (local address 127.0.0.1:57178)
0925 13:07:28.460481 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:28.460502 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260925 13:07:28.473233 5390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.473020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:39063 (local address 127.0.0.1:50102)
0925 13:07:28.473174 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:28.473203 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.67:39063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260925 13:07:28.473239 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 4892
2026-09-25T13:07:28Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2089 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-25T13:07:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:28Z Disabled control of system clock
I20260925 13:07:28.500624 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:45343
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:35859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:45343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 13:07:28.580387 5426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:28.580613 5426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:28.580648 5426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:28.581759 5426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:28.581820 5426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:28.581851 5426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:28.581883 5426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:28.581908 5426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:07:28.583243 5426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:35859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:45343
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:45343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:28.583536 5426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:28.583824 5426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:28.588431 5432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:28.588456 5431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:28.588639 5434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:28.588682 5426 server_base.cc:1034] running on GCE node
I20260925 13:07:28.588917 5426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:28.589224 5426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:28.590384 5426 hybrid_clock.cc:648] HybridClock initialized: now 1790341648590369 us; error 30 us; skew 500 ppm
I20260925 13:07:28.591595 5426 webserver.cc:460] Webserver started at http://127.0.206.126:39407/ using document root <none> and password file <none>
I20260925 13:07:28.591820 5426 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:28.591897 5426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:28.592043 5426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:28.593122 5426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "2c50b668f8954a018d8e38f921f577ef"
format_stamp: "Formatted at 2026-09-25 13:07:28 on dist-test-slave-chkf"
server_key: "ce6183e8e0f673734489ac5b0dfe70c9"
server_key_iv: "2aa275e4d4e7b56f4ef3e613a72d4908"
server_key_version: "encryptionkey@0"
I20260925 13:07:28.593464 5426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "2c50b668f8954a018d8e38f921f577ef"
format_stamp: "Formatted at 2026-09-25 13:07:28 on dist-test-slave-chkf"
server_key: "ce6183e8e0f673734489ac5b0dfe70c9"
server_key_iv: "2aa275e4d4e7b56f4ef3e613a72d4908"
server_key_version: "encryptionkey@0"
I20260925 13:07:28.594789 5426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:07:28.595475 5440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:28.595634 5426 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:28.595729 5426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "2c50b668f8954a018d8e38f921f577ef"
format_stamp: "Formatted at 2026-09-25 13:07:28 on dist-test-slave-chkf"
server_key: "ce6183e8e0f673734489ac5b0dfe70c9"
server_key_iv: "2aa275e4d4e7b56f4ef3e613a72d4908"
server_key_version: "encryptionkey@0"
I20260925 13:07:28.595798 5426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:28.599032 5426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:28.599278 5426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:28.599411 5426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:28.603166 5426 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:45343
I20260925 13:07:28.603202 5492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:45343 every 8 connection(s)
I20260925 13:07:28.603500 5426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:28.604080 5493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:28.605021 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 5426
I20260925 13:07:28.605134 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:07:28.605290 825 external_mini_cluster.cc:1428] Setting key e44ba9c2cadc59596ea3867127d45ae3
I20260925 13:07:28.607694 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.605762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39614 (local address 127.0.206.126:45343)
0925 13:07:28.605946 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:28.605955 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:28.605984 (+ 29us) server_negotiation.cc:407] Connection header received
0925 13:07:28.606436 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:28.606441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:28.606483 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:28.606617 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:28.607282 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:28.607285 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:28.607286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:07:28.607289 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:28.607335 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:07:28.607345 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:28.607429 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:28.607497 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 13:07:28.607552 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260925 13:07:28.608603 5493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef: Bootstrap starting.
I20260925 13:07:28.609153 5493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:28.609418 5493 log.cc:824] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:28.610059 5493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef: No bootstrap required, opened a new log
I20260925 13:07:28.611399 5493 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c50b668f8954a018d8e38f921f577ef" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45343 } }
I20260925 13:07:28.611527 5493 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:28.611549 5493 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c50b668f8954a018d8e38f921f577ef, State: Initialized, Role: FOLLOWER
I20260925 13:07:28.611680 5493 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c50b668f8954a018d8e38f921f577ef" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45343 } }
I20260925 13:07:28.611775 5493 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:28.611819 5493 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:28.611874 5493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:28.673206 5493 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c50b668f8954a018d8e38f921f577ef" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45343 } }
I20260925 13:07:28.673360 5493 leader_election.cc:302] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c50b668f8954a018d8e38f921f577ef; no voters:
I20260925 13:07:28.673580 5493 leader_election.cc:288] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:07:28.673668 5498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:28.673828 5498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef [term 1 LEADER]: Becoming Leader. State: Replica: 2c50b668f8954a018d8e38f921f577ef, State: Running, Role: LEADER
I20260925 13:07:28.673871 5493 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:28.673934 5498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c50b668f8954a018d8e38f921f577ef" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45343 } }
I20260925 13:07:28.674288 5499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c50b668f8954a018d8e38f921f577ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c50b668f8954a018d8e38f921f577ef" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45343 } } }
I20260925 13:07:28.674407 5499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:28.674929 5505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:28.674674 5500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c50b668f8954a018d8e38f921f577ef. Latest consensus state: current_term: 1 leader_uuid: "2c50b668f8954a018d8e38f921f577ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c50b668f8954a018d8e38f921f577ef" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45343 } } }
I20260925 13:07:28.675143 5500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:28.675719 5505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:07:28.678056 5505 catalog_manager.cc:1347] Generated new cluster ID: ef8c700fb628439e960e892be94bd625
I20260925 13:07:28.678115 5505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:07:28.690585 5505 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:07:28.690785 5505 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:28.699450 5505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef: Generated new TSK 0
I20260925 13:07:28.699635 5505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:07:28.708294 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45343
--builtin_ntp_servers=127.0.206.84:35859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 13:07:28.791085 5517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:28.791283 5517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:28.791306 5517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:28.792549 5517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:28.792585 5517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:28.792611 5517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:07:28.793980 5517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:35859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:28.794207 5517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:28.794505 5517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:28.800402 5525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:28.800421 5523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:28.800551 5522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:28.800537 5517 server_base.cc:1034] running on GCE node
I20260925 13:07:28.800738 5517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:28.801016 5517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:28.802168 5517 hybrid_clock.cc:648] HybridClock initialized: now 1790341648802146 us; error 37 us; skew 500 ppm
I20260925 13:07:28.803390 5517 webserver.cc:460] Webserver started at http://127.0.206.65:37503/ using document root <none> and password file <none>
I20260925 13:07:28.803632 5517 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:28.803676 5517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:28.803818 5517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:28.804853 5517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "432f0628623647e88bc64379d6a8c970"
format_stamp: "Formatted at 2026-09-25 13:07:28 on dist-test-slave-chkf"
server_key: "6b3c570feabe2ebf1a316f26af33f4e8"
server_key_iv: "13d9b6b98d60e919bfc4a844210044c7"
server_key_version: "encryptionkey@0"
I20260925 13:07:28.805210 5517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "432f0628623647e88bc64379d6a8c970"
format_stamp: "Formatted at 2026-09-25 13:07:28 on dist-test-slave-chkf"
server_key: "6b3c570feabe2ebf1a316f26af33f4e8"
server_key_iv: "13d9b6b98d60e919bfc4a844210044c7"
server_key_version: "encryptionkey@0"
I20260925 13:07:28.806476 5517 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:07:28.807229 5531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:28.807400 5517 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:28.807469 5517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "432f0628623647e88bc64379d6a8c970"
format_stamp: "Formatted at 2026-09-25 13:07:28 on dist-test-slave-chkf"
server_key: "6b3c570feabe2ebf1a316f26af33f4e8"
server_key_iv: "13d9b6b98d60e919bfc4a844210044c7"
server_key_version: "encryptionkey@0"
I20260925 13:07:28.807567 5517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:28.810820 5517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:28.811095 5517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:28.811225 5517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:28.811434 5517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:28.811718 5517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:28.811748 5517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:28.811801 5517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:28.811830 5517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:28.817459 5517 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:46143
I20260925 13:07:28.817502 5644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:46143 every 8 connection(s)
I20260925 13:07:28.817801 5517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:07:28.820803 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.818486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:43683 (local address 127.0.206.126:45343)
0925 13:07:28.818554 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:28.818557 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:28.818798 (+ 241us) server_negotiation.cc:407] Connection header received
0925 13:07:28.819331 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:28.819332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:28.819339 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:28.819403 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:28.820560 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:28.820561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:28.820562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:07:28.820567 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:28.820581 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:07:28.820585 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:28.820637 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:28.820718 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 13:07:28.820733 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 13:07:28.821002 5647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.818487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45343 (local address 127.0.206.65:43683)
0925 13:07:28.818661 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:28.818699 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:28.819228 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:28.819473 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:28.819477 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:28.819487 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:28.820417 (+ 930us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:07:28.820417 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:28.820444 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:07:28.820472 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:28.820642 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:28.820646 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:28.820649 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:07:28.820723 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 13:07:28.820783 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260925 13:07:28.821455 5645 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45343
I20260925 13:07:28.821550 5645 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:28.821703 5645 heartbeater.cc:511] Master 127.0.206.126:45343 requested a full tablet report, sending...
I20260925 13:07:28.822175 5457 ts_manager.cc:194] Registered new tserver with Master: 432f0628623647e88bc64379d6a8c970 (127.0.206.65:46143)
I20260925 13:07:28.822952 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 5517
I20260925 13:07:28.823066 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:07:28.823239 825 external_mini_cluster.cc:1428] Setting key 41167d25c0940495301b450c8519dec2
I20260925 13:07:28.824123 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45343
--builtin_ntp_servers=127.0.206.84:35859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 13:07:28.903517 5648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:28.903731 5648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:28.903769 5648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:28.904906 5648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:28.904964 5648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:28.905009 5648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:07:28.906309 5648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:35859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:28.906554 5648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:28.906862 5648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:28.911576 5654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:28.911583 5653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:28.911592 5656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:28.911824 5648 server_base.cc:1034] running on GCE node
I20260925 13:07:28.912038 5648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:28.912274 5648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:28.913429 5648 hybrid_clock.cc:648] HybridClock initialized: now 1790341648913416 us; error 28 us; skew 500 ppm
I20260925 13:07:28.914603 5648 webserver.cc:460] Webserver started at http://127.0.206.66:46869/ using document root <none> and password file <none>
I20260925 13:07:28.914865 5648 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:28.914937 5648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:28.915068 5648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:28.916079 5648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "02a15561b2954023abe351fd0542b193"
format_stamp: "Formatted at 2026-09-25 13:07:28 on dist-test-slave-chkf"
server_key: "fc38cc046f0607d519cfc6ebe53660f2"
server_key_iv: "0c23c23ea2fc6805a5abc0cab36d9422"
server_key_version: "encryptionkey@0"
I20260925 13:07:28.916429 5648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "02a15561b2954023abe351fd0542b193"
format_stamp: "Formatted at 2026-09-25 13:07:28 on dist-test-slave-chkf"
server_key: "fc38cc046f0607d519cfc6ebe53660f2"
server_key_iv: "0c23c23ea2fc6805a5abc0cab36d9422"
server_key_version: "encryptionkey@0"
I20260925 13:07:28.917670 5648 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:07:28.918382 5662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:28.918545 5648 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:28.918634 5648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "02a15561b2954023abe351fd0542b193"
format_stamp: "Formatted at 2026-09-25 13:07:28 on dist-test-slave-chkf"
server_key: "fc38cc046f0607d519cfc6ebe53660f2"
server_key_iv: "0c23c23ea2fc6805a5abc0cab36d9422"
server_key_version: "encryptionkey@0"
I20260925 13:07:28.918751 5648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:28.921931 5648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:28.922163 5648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:28.922295 5648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:28.922503 5648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:28.922849 5648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:28.922909 5648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:28.922941 5648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:28.922963 5648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:28.928539 5648 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:35773
I20260925 13:07:28.928571 5775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:35773 every 8 connection(s)
I20260925 13:07:28.928895 5648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:07:28.931767 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.929602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:52487 (local address 127.0.206.126:45343)
0925 13:07:28.929659 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:28.929661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:28.929875 (+ 214us) server_negotiation.cc:407] Connection header received
0925 13:07:28.930349 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:28.930350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:28.930356 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:28.930422 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:28.931524 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:28.931525 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:28.931525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:07:28.931530 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:28.931543 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:07:28.931547 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:28.931602 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:28.931684 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 13:07:28.931698 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 13:07:28.931946 5778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:28.929588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45343 (local address 127.0.206.66:52487)
0925 13:07:28.929766 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:28.929797 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:28.930254 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:28.930499 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:28.930502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:28.930510 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:28.931380 (+ 870us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:07:28.931381 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:28.931407 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:07:28.931434 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:28.931602 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:28.931607 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:28.931609 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:07:28.931694 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 13:07:28.931751 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260925 13:07:28.932384 5776 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45343
I20260925 13:07:28.932502 5776 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:28.932646 5776 heartbeater.cc:511] Master 127.0.206.126:45343 requested a full tablet report, sending...
I20260925 13:07:28.932976 5457 ts_manager.cc:194] Registered new tserver with Master: 02a15561b2954023abe351fd0542b193 (127.0.206.66:35773)
I20260925 13:07:28.938341 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 5648
I20260925 13:07:28.938431 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:07:28.938601 825 external_mini_cluster.cc:1428] Setting key d612e62e452c2dff33e5ecc1cf1c4ad8
I20260925 13:07:28.939519 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45343
--builtin_ntp_servers=127.0.206.84:35859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 13:07:29.024060 5779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:29.024252 5779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:29.024274 5779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:29.025441 5779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:29.025480 5779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:29.025506 5779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:07:29.026916 5779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:35859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:29.027166 5779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:29.027453 5779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:29.033802 5785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:29.033896 5787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:29.033794 5784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:29.034188 5779 server_base.cc:1034] running on GCE node
I20260925 13:07:29.034366 5779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:29.034617 5779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:29.035760 5779 hybrid_clock.cc:648] HybridClock initialized: now 1790341649035748 us; error 27 us; skew 500 ppm
I20260925 13:07:29.036944 5779 webserver.cc:460] Webserver started at http://127.0.206.67:38685/ using document root <none> and password file <none>
I20260925 13:07:29.037184 5779 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:29.037227 5779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:29.037360 5779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:29.038415 5779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "44b51610df9349658e90944f7c337966"
format_stamp: "Formatted at 2026-09-25 13:07:29 on dist-test-slave-chkf"
server_key: "0b8147575c84f07120ba9df883399d6e"
server_key_iv: "b1d0f33bc6a436b6a788b8d273c75433"
server_key_version: "encryptionkey@0"
I20260925 13:07:29.038825 5779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "44b51610df9349658e90944f7c337966"
format_stamp: "Formatted at 2026-09-25 13:07:29 on dist-test-slave-chkf"
server_key: "0b8147575c84f07120ba9df883399d6e"
server_key_iv: "b1d0f33bc6a436b6a788b8d273c75433"
server_key_version: "encryptionkey@0"
I20260925 13:07:29.040145 5779 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:29.040889 5793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:29.041038 5779 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:29.041118 5779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "44b51610df9349658e90944f7c337966"
format_stamp: "Formatted at 2026-09-25 13:07:29 on dist-test-slave-chkf"
server_key: "0b8147575c84f07120ba9df883399d6e"
server_key_iv: "b1d0f33bc6a436b6a788b8d273c75433"
server_key_version: "encryptionkey@0"
I20260925 13:07:29.041183 5779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:29.044549 5779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:29.044823 5779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:29.044950 5779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:29.045173 5779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:29.045511 5779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:29.045547 5779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:29.045567 5779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:29.045607 5779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:29.052510 5779 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:42801
I20260925 13:07:29.052551 5906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:42801 every 8 connection(s)
I20260925 13:07:29.052848 5779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:07:29.053968 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 5779
I20260925 13:07:29.054082 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:07:29.054250 825 external_mini_cluster.cc:1428] Setting key 21ab6d7d76aeda5b0a90b7d2a913b744
I20260925 13:07:29.056169 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.053554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:34513 (local address 127.0.206.126:45343)
0925 13:07:29.053617 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:29.053619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:29.053871 (+ 252us) server_negotiation.cc:407] Connection header received
0925 13:07:29.054553 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:29.054555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:29.054561 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:29.054625 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:29.055906 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:29.055907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:29.055908 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:07:29.055913 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:29.055931 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:07:29.055937 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:29.056003 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:29.056057 (+ 54us) server_negotiation.cc:299] Negotiation successful
0925 13:07:29.056072 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 13:07:29.056350 5909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.053539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45343 (local address 127.0.206.67:34513)
0925 13:07:29.053729 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:29.053768 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:29.054428 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:29.054692 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:29.054695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:29.054732 (+ 37us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:29.055759 (+ 1027us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:07:29.055759 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:29.055788 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:07:29.055817 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:29.056002 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:29.056004 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:29.056007 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:07:29.056059 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 13:07:29.056106 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260925 13:07:29.056715 5907 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45343
I20260925 13:07:29.056828 5907 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:29.056978 5907 heartbeater.cc:511] Master 127.0.206.126:45343 requested a full tablet report, sending...
I20260925 13:07:29.057298 5457 ts_manager.cc:194] Registered new tserver with Master: 44b51610df9349658e90944f7c337966 (127.0.206.67:42801)
I20260925 13:07:29.066041 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:07:29.069340 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.067504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39620 (local address 127.0.206.126:45343)
0925 13:07:29.067562 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:29.067565 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:29.067636 (+ 71us) server_negotiation.cc:407] Connection header received
0925 13:07:29.067693 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:29.067694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:29.067701 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:29.067792 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:29.069095 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:29.069096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:29.069097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:07:29.069101 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:29.069116 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:07:29.069121 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:29.069191 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:29.069226 (+ 35us) server_negotiation.cc:299] Negotiation successful
0925 13:07:29.069241 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 13:07:29.071135 5457 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:07:29.073945 5457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:07:29.077891 5915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.076219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:35773 (local address 127.0.0.1:56528)
0925 13:07:29.076339 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:29.076383 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:29.076440 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:29.076828 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:29.076833 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:29.076840 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:29.077396 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:07:29.077397 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:29.077412 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:07:29.077437 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:29.077734 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:29.077740 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:29.077742 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:07:29.077790 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 13:07:29.077803 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":60,"threads_started":1}
I20260925 13:07:29.078059 5919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.076841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34580 (local address 127.0.206.67:42801)
0925 13:07:29.077030 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:29.077034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:29.077044 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:07:29.077069 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:29.077072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:29.077080 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:29.077321 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:29.077723 (+ 402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:29.077728 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:29.077728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:07:29.077732 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:29.077759 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:07:29.077776 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:29.077931 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:29.077954 (+ 23us) server_negotiation.cc:299] Negotiation successful
0925 13:07:29.077969 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":98,"threads_started":1}
I20260925 13:07:29.078284 5917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.076414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:42801 (local address 127.0.0.1:34580)
0925 13:07:29.076527 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:29.076535 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:29.076567 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:29.077178 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:29.077179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:29.077182 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:29.077627 (+ 445us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:07:29.077628 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:29.077634 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:07:29.077643 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:29.077847 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:29.077848 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:29.077848 (+ 0us) client_negotiation.cc:769] Sending connection context
0925 13:07:29.077874 (+ 26us) client_negotiation.cc:240] Negotiation successful
0925 13:07:29.077884 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
I20260925 13:07:29.078524 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.076306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56528 (local address 127.0.206.66:35773)
0925 13:07:29.076671 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:29.076676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:29.076687 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:07:29.076714 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:29.076716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:29.076729 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:29.077089 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:29.077600 (+ 511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:29.077605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:29.077606 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:07:29.077610 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:29.077642 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:07:29.077658 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:29.078404 (+ 746us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:29.078430 (+ 26us) server_negotiation.cc:299] Negotiation successful
0925 13:07:29.078445 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":111,"threads_started":1}
I20260925 13:07:29.079722 5841 tablet_service.cc:1467] Processing CreateTablet for tablet a1dcc8a7f1e14d0ba03d4cf2a7519fab (DEFAULT_TABLE table=test-table [id=c1e2724921ca42328dd505af0e424eb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:29.079722 5710 tablet_service.cc:1467] Processing CreateTablet for tablet a1dcc8a7f1e14d0ba03d4cf2a7519fab (DEFAULT_TABLE table=test-table [id=c1e2724921ca42328dd505af0e424eb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:29.080191 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1dcc8a7f1e14d0ba03d4cf2a7519fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:29.080191 5710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1dcc8a7f1e14d0ba03d4cf2a7519fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:29.083741 5921 tablet_bootstrap.cc:492] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966: Bootstrap starting.
I20260925 13:07:29.083833 5918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.076559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:46143 (local address 127.0.0.1:36076)
0925 13:07:29.076702 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:29.076711 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:29.076748 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:29.082606 (+ 5858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:29.082607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:29.082610 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:29.083148 (+ 538us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:07:29.083148 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:29.083157 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:07:29.083171 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:29.083686 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:29.083688 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:29.083689 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 13:07:29.083721 (+ 32us) client_negotiation.cc:240] Negotiation successful
0925 13:07:29.083735 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260925 13:07:29.084172 5920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.077020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36076 (local address 127.0.206.65:46143)
0925 13:07:29.082424 (+ 5404us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:29.082429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:29.082443 (+ 14us) server_negotiation.cc:407] Connection header received
0925 13:07:29.082470 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:29.082473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:29.082484 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:29.083255 (+ 771us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:29.083548 (+ 293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:29.083552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:29.083552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:07:29.083555 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:29.083587 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:07:29.083602 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:29.084011 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:29.084039 (+ 28us) server_negotiation.cc:299] Negotiation successful
0925 13:07:29.084057 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5357,"thread_start_us":1781,"threads_started":1}
I20260925 13:07:29.084431 5921 tablet_bootstrap.cc:654] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:29.084432 5922 tablet_bootstrap.cc:492] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193: Bootstrap starting.
I20260925 13:07:29.084748 5921 log.cc:824] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:29.085047 5579 tablet_service.cc:1467] Processing CreateTablet for tablet a1dcc8a7f1e14d0ba03d4cf2a7519fab (DEFAULT_TABLE table=test-table [id=c1e2724921ca42328dd505af0e424eb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:29.085312 5922 tablet_bootstrap.cc:654] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:29.085391 5579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1dcc8a7f1e14d0ba03d4cf2a7519fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:29.085511 5921 tablet_bootstrap.cc:492] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966: No bootstrap required, opened a new log
I20260925 13:07:29.085568 5921 ts_tablet_manager.cc:1397] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:07:29.085873 5922 log.cc:824] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:29.087473 5922 tablet_bootstrap.cc:492] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193: No bootstrap required, opened a new log
I20260925 13:07:29.087536 5922 ts_tablet_manager.cc:1397] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 13:07:29.088268 5921 raft_consensus.cc:348] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02a15561b2954023abe351fd0542b193" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 35773 } } peers { permanent_uuid: "44b51610df9349658e90944f7c337966" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42801 } } peers { permanent_uuid: "432f0628623647e88bc64379d6a8c970" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 46143 } }
I20260925 13:07:29.088435 5921 raft_consensus.cc:374] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:29.088471 5921 raft_consensus.cc:729] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44b51610df9349658e90944f7c337966, State: Initialized, Role: FOLLOWER
I20260925 13:07:29.088608 5921 consensus_queue.cc:260] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02a15561b2954023abe351fd0542b193" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 35773 } } peers { permanent_uuid: "44b51610df9349658e90944f7c337966" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42801 } } peers { permanent_uuid: "432f0628623647e88bc64379d6a8c970" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 46143 } }
I20260925 13:07:29.088842 5921 ts_tablet_manager.cc:1428] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260925 13:07:29.088964 5907 heartbeater.cc:503] Master 127.0.206.126:45343 was elected leader, sending a full tablet report...
I20260925 13:07:29.090155 5922 raft_consensus.cc:348] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02a15561b2954023abe351fd0542b193" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 35773 } } peers { permanent_uuid: "44b51610df9349658e90944f7c337966" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42801 } } peers { permanent_uuid: "432f0628623647e88bc64379d6a8c970" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 46143 } }
I20260925 13:07:29.090304 5922 raft_consensus.cc:374] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:29.090341 5922 raft_consensus.cc:729] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02a15561b2954023abe351fd0542b193, State: Initialized, Role: FOLLOWER
I20260925 13:07:29.090456 5922 consensus_queue.cc:260] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02a15561b2954023abe351fd0542b193" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 35773 } } peers { permanent_uuid: "44b51610df9349658e90944f7c337966" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42801 } } peers { permanent_uuid: "432f0628623647e88bc64379d6a8c970" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 46143 } }
I20260925 13:07:29.090653 5922 ts_tablet_manager.cc:1428] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 13:07:29.090755 5776 heartbeater.cc:503] Master 127.0.206.126:45343 was elected leader, sending a full tablet report...
I20260925 13:07:29.094871 5926 tablet_bootstrap.cc:492] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 432f0628623647e88bc64379d6a8c970: Bootstrap starting.
I20260925 13:07:29.095424 5926 tablet_bootstrap.cc:654] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 432f0628623647e88bc64379d6a8c970: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:29.095733 5926 log.cc:824] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 432f0628623647e88bc64379d6a8c970: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:29.096438 5926 tablet_bootstrap.cc:492] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 432f0628623647e88bc64379d6a8c970: No bootstrap required, opened a new log
I20260925 13:07:29.096513 5926 ts_tablet_manager.cc:1397] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 432f0628623647e88bc64379d6a8c970: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:07:29.098326 5926 raft_consensus.cc:348] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 432f0628623647e88bc64379d6a8c970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02a15561b2954023abe351fd0542b193" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 35773 } } peers { permanent_uuid: "44b51610df9349658e90944f7c337966" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42801 } } peers { permanent_uuid: "432f0628623647e88bc64379d6a8c970" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 46143 } }
I20260925 13:07:29.098515 5926 raft_consensus.cc:374] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 432f0628623647e88bc64379d6a8c970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:29.098559 5926 raft_consensus.cc:729] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 432f0628623647e88bc64379d6a8c970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 432f0628623647e88bc64379d6a8c970, State: Initialized, Role: FOLLOWER
I20260925 13:07:29.098677 5926 consensus_queue.cc:260] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 432f0628623647e88bc64379d6a8c970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02a15561b2954023abe351fd0542b193" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 35773 } } peers { permanent_uuid: "44b51610df9349658e90944f7c337966" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42801 } } peers { permanent_uuid: "432f0628623647e88bc64379d6a8c970" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 46143 } }
I20260925 13:07:29.098944 5926 ts_tablet_manager.cc:1428] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 432f0628623647e88bc64379d6a8c970: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 13:07:29.099017 5645 heartbeater.cc:503] Master 127.0.206.126:45343 was elected leader, sending a full tablet report...
W20260925 13:07:29.179426 5777 tablet.cc:2369] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:07:29.291271 5925 raft_consensus.cc:482] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:29.291435 5925 raft_consensus.cc:504] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02a15561b2954023abe351fd0542b193" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 35773 } } peers { permanent_uuid: "44b51610df9349658e90944f7c337966" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42801 } } peers { permanent_uuid: "432f0628623647e88bc64379d6a8c970" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 46143 } }
I20260925 13:07:29.291834 5925 leader_election.cc:288] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02a15561b2954023abe351fd0542b193 (127.0.206.66:35773), 432f0628623647e88bc64379d6a8c970 (127.0.206.65:46143)
I20260925 13:07:29.293390 5909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.292165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:35773 (local address 127.0.206.67:60359)
0925 13:07:29.292222 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:29.292236 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:29.292298 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:29.292504 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:29.292505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:29.292508 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:29.293046 (+ 538us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:07:29.293046 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:29.293054 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:07:29.293068 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:29.293232 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:29.293233 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:29.293234 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 13:07:29.293293 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 13:07:29.293307 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 13:07:29.293478 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.292260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:60359 (local address 127.0.206.66:35773)
0925 13:07:29.292314 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:29.292316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:29.292327 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:07:29.292357 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:29.292358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:29.292363 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:29.292445 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:29.293153 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:29.293154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:29.293154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:07:29.293159 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:29.293174 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:07:29.293179 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:29.293231 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:29.293374 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 13:07:29.293408 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 13:07:29.293489 5920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.292480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:54365 (local address 127.0.206.65:46143)
0925 13:07:29.292538 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:29.292541 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:29.292588 (+ 47us) server_negotiation.cc:407] Connection header received
0925 13:07:29.292696 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:29.292697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:29.292703 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:29.292755 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:29.293294 (+ 539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:29.293295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:29.293295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:07:29.293299 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:29.293311 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:07:29.293314 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:29.293354 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:29.293417 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 13:07:29.293430 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 13:07:29.293507 5930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.292328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:46143 (local address 127.0.206.67:54365)
0925 13:07:29.292520 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:29.292530 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:29.292617 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:29.292770 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:29.292771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:29.292786 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:29.293207 (+ 421us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:07:29.293207 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:29.293217 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:07:29.293232 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:29.293352 (+ 120us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:29.293353 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:29.293353 (+ 0us) client_negotiation.cc:769] Sending connection context
0925 13:07:29.293400 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 13:07:29.293419 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":121,"threads_started":1}
I20260925 13:07:29.293844 5730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1dcc8a7f1e14d0ba03d4cf2a7519fab" candidate_uuid: "44b51610df9349658e90944f7c337966" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02a15561b2954023abe351fd0542b193" is_pre_election: true
I20260925 13:07:29.294018 5730 raft_consensus.cc:2454] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44b51610df9349658e90944f7c337966 in term 0.
I20260925 13:07:29.294061 5599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1dcc8a7f1e14d0ba03d4cf2a7519fab" candidate_uuid: "44b51610df9349658e90944f7c337966" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "432f0628623647e88bc64379d6a8c970" is_pre_election: true
I20260925 13:07:29.294199 5599 raft_consensus.cc:2454] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 432f0628623647e88bc64379d6a8c970 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44b51610df9349658e90944f7c337966 in term 0.
I20260925 13:07:29.294288 5797 leader_election.cc:302] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02a15561b2954023abe351fd0542b193, 44b51610df9349658e90944f7c337966; no voters:
I20260925 13:07:29.294440 5925 raft_consensus.cc:2781] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:07:29.294513 5925 raft_consensus.cc:482] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:07:29.294531 5925 raft_consensus.cc:3037] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:07:29.303354 5908 tablet.cc:2369] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:07:29.318547 5646 tablet.cc:2369] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 432f0628623647e88bc64379d6a8c970: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:07:29.392019 5925 raft_consensus.cc:504] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02a15561b2954023abe351fd0542b193" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 35773 } } peers { permanent_uuid: "44b51610df9349658e90944f7c337966" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42801 } } peers { permanent_uuid: "432f0628623647e88bc64379d6a8c970" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 46143 } }
I20260925 13:07:29.392218 5925 leader_election.cc:288] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [CANDIDATE]: Term 1 election: Requested vote from peers 02a15561b2954023abe351fd0542b193 (127.0.206.66:35773), 432f0628623647e88bc64379d6a8c970 (127.0.206.65:46143)
I20260925 13:07:29.392562 5730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1dcc8a7f1e14d0ba03d4cf2a7519fab" candidate_uuid: "44b51610df9349658e90944f7c337966" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02a15561b2954023abe351fd0542b193"
I20260925 13:07:29.392613 5599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1dcc8a7f1e14d0ba03d4cf2a7519fab" candidate_uuid: "44b51610df9349658e90944f7c337966" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "432f0628623647e88bc64379d6a8c970"
I20260925 13:07:29.392679 5730 raft_consensus.cc:3037] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:29.392731 5599 raft_consensus.cc:3037] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 432f0628623647e88bc64379d6a8c970 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:29.393455 5730 raft_consensus.cc:2454] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44b51610df9349658e90944f7c337966 in term 1.
I20260925 13:07:29.393677 5797 leader_election.cc:302] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02a15561b2954023abe351fd0542b193, 44b51610df9349658e90944f7c337966; no voters:
I20260925 13:07:29.393796 5925 raft_consensus.cc:2781] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:29.393812 5599 raft_consensus.cc:2454] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 432f0628623647e88bc64379d6a8c970 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44b51610df9349658e90944f7c337966 in term 1.
I20260925 13:07:29.394016 5925 raft_consensus.cc:686] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [term 1 LEADER]: Becoming Leader. State: Replica: 44b51610df9349658e90944f7c337966, State: Running, Role: LEADER
I20260925 13:07:29.394124 5925 consensus_queue.cc:237] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02a15561b2954023abe351fd0542b193" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 35773 } } peers { permanent_uuid: "44b51610df9349658e90944f7c337966" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42801 } } peers { permanent_uuid: "432f0628623647e88bc64379d6a8c970" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 46143 } }
I20260925 13:07:29.395023 5457 catalog_manager.cc:5495] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44b51610df9349658e90944f7c337966 (127.0.206.67). New cstate: current_term: 1 leader_uuid: "44b51610df9349658e90944f7c337966" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02a15561b2954023abe351fd0542b193" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 35773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44b51610df9349658e90944f7c337966" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "432f0628623647e88bc64379d6a8c970" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 46143 } health_report { overall_health: UNKNOWN } } }
I20260925 13:07:29.441529 5919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.440353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34584 (local address 127.0.206.67:42801)
0925 13:07:29.440461 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:29.440463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:29.440474 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:07:29.440518 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:29.440519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:29.440538 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:29.440603 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:29.441260 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:29.441262 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:29.441262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:07:29.441266 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:29.441297 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:07:29.441301 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:29.441347 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:29.441411 (+ 64us) server_negotiation.cc:299] Negotiation successful
0925 13:07:29.441426 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 13:07:29.443459 5599 raft_consensus.cc:1261] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 432f0628623647e88bc64379d6a8c970 [term 1 FOLLOWER]: Refusing update from remote peer 44b51610df9349658e90944f7c337966: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:29.443480 5730 raft_consensus.cc:1261] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193 [term 1 FOLLOWER]: Refusing update from remote peer 44b51610df9349658e90944f7c337966: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:29.443818 5931 consensus_queue.cc:1035] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [LEADER]: Connected to new peer: Peer: permanent_uuid: "432f0628623647e88bc64379d6a8c970" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 46143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:29.443943 5925 consensus_queue.cc:1035] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02a15561b2954023abe351fd0542b193" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 35773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:29.446162 5934 mvcc.cc:205] Tried to move back new op lower bound from 7333239396117819392 to 7333239395918979072. Current Snapshot: MvccSnapshot[applied={T|T < 7333239396117819392}]
I20260925 13:07:29.446662 5935 mvcc.cc:205] Tried to move back new op lower bound from 7333239396117819392 to 7333239395918979072. Current Snapshot: MvccSnapshot[applied={T|T < 7333239396117819392}]
I20260925 13:07:29.448974 5457 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:29.449059 5457 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:29.449753 5457 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2c50b668f8954a018d8e38f921f577ef: Sending DeleteTablet for 3 replicas of tablet a1dcc8a7f1e14d0ba03d4cf2a7519fab
I20260925 13:07:29.450062 5710 tablet_service.cc:1514] Processing DeleteTablet for tablet a1dcc8a7f1e14d0ba03d4cf2a7519fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:29 UTC) from {username='slave'} at 127.0.0.1:56528
I20260925 13:07:29.450081 5841 tablet_service.cc:1514] Processing DeleteTablet for tablet a1dcc8a7f1e14d0ba03d4cf2a7519fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:29 UTC) from {username='slave'} at 127.0.0.1:34580
I20260925 13:07:29.450181 5579 tablet_service.cc:1514] Processing DeleteTablet for tablet a1dcc8a7f1e14d0ba03d4cf2a7519fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:29 UTC) from {username='slave'} at 127.0.0.1:36076
I20260925 13:07:29.450393 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 5517
I20260925 13:07:29.450398 5943 tablet_replica.cc:330] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193: stopping tablet replica
I20260925 13:07:29.450527 5944 tablet_replica.cc:330] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966: stopping tablet replica
I20260925 13:07:29.450630 5944 raft_consensus.cc:2229] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:07:29.450745 5943 raft_consensus.cc:2229] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:29.450891 5944 raft_consensus.cc:2258] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:29.451022 5943 raft_consensus.cc:2258] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:29.451778 5944 ts_tablet_manager.cc:1905] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:29.451997 5943 ts_tablet_manager.cc:1905] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 02a15561b2954023abe351fd0542b193: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:07:29.456049 5444 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:46143 (error 108)
I20260925 13:07:29.456517 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 5648
I20260925 13:07:29.463140 5918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.462990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:46143 (local address 127.0.0.1:36092)
0925 13:07:29.463052 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:29.463101 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:46143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260925 13:07:29.463306 5444 catalog_manager.cc:4584] TS 432f0628623647e88bc64379d6a8c970 (127.0.206.65:46143): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a1dcc8a7f1e14d0ba03d4cf2a7519fab: Network error: Client connection negotiation failed: client connection to 127.0.206.65:46143: connect: Connection refused (error 111)
I20260925 13:07:29.494076 5944 ts_tablet_manager.cc:1918] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:07:29.494266 5944 log.cc:1196] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/a1dcc8a7f1e14d0ba03d4cf2a7519fab
I20260925 13:07:29.494657 5944 ts_tablet_manager.cc:1939] T a1dcc8a7f1e14d0ba03d4cf2a7519fab P 44b51610df9349658e90944f7c337966: Deleting consensus metadata
I20260925 13:07:29.497304 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 5779
I20260925 13:07:29.497393 5918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.497264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:35773 (local address 127.0.0.1:56542)
0925 13:07:29.497328 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:29.497353 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.66:35773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 13:07:29.516335 5918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.516161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:46143 (local address 127.0.0.1:36094)
0925 13:07:29.516238 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:29.516296 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:46143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 13:07:29.516763 5918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.516665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:46143 (local address 127.0.0.1:36104)
0925 13:07:29.516709 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:29.516740 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:46143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260925 13:07:29.520577 5918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.520470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:42801 (local address 127.0.0.1:34586)
0925 13:07:29.520516 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:29.520543 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.67:42801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 13:07:29.520593 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 5426
2026-09-25T13:07:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1046 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260925 13:07:29.540304 825 mini_oidc.cc:152] Starting JWKS server
I20260925 13:07:29.541376 825 webserver.cc:460] Webserver started at https://127.0.0.1:35113/ using document root <none> and password file <none>
I20260925 13:07:29.541478 825 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 13:07:29.541647 825 webserver.cc:460] Webserver started at http://127.0.0.1:35289/ using document root <none> and password file <none>
2026-09-25T13:07:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:29Z Disabled control of system clock
I20260925 13:07:29.548970 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:35327
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:33507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:35327
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35113/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/testchainca.cert with env {}
W20260925 13:07:29.627681 5952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:29.627903 5952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:29.627925 5952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:29.629024 5952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:29.629060 5952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:29.629072 5952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:29.629084 5952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:29.629096 5952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 13:07:29.629108 5952 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 13:07:29.629125 5952 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35113/jwks.json
I20260925 13:07:29.630549 5952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:35327
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:35327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35113/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:29.630818 5952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:29.630990 5952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:29.635528 5960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:29.635599 5952 server_base.cc:1034] running on GCE node
W20260925 13:07:29.635524 5958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:29.635528 5957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:29.636034 5952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:29.636337 5952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:29.637491 5952 hybrid_clock.cc:648] HybridClock initialized: now 1790341649637484 us; error 35 us; skew 500 ppm
I20260925 13:07:29.638834 5952 webserver.cc:460] Webserver started at http://127.0.206.126:39773/ using document root <none> and password file <none>
I20260925 13:07:29.639070 5952 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:29.639135 5952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:29.639267 5952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:29.640297 5952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "c17c6b2ee2fc4b45807635fc87613ad5"
format_stamp: "Formatted at 2026-09-25 13:07:29 on dist-test-slave-chkf"
server_key: "ede2dfb47971ac8ce8df595cc3331e95"
server_key_iv: "2611758ae84a0731b1abd445c8ee2447"
server_key_version: "encryptionkey@0"
I20260925 13:07:29.640697 5952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "c17c6b2ee2fc4b45807635fc87613ad5"
format_stamp: "Formatted at 2026-09-25 13:07:29 on dist-test-slave-chkf"
server_key: "ede2dfb47971ac8ce8df595cc3331e95"
server_key_iv: "2611758ae84a0731b1abd445c8ee2447"
server_key_version: "encryptionkey@0"
I20260925 13:07:29.642095 5952 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:07:29.642870 5966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:29.643050 5952 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:07:29.643143 5952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "c17c6b2ee2fc4b45807635fc87613ad5"
format_stamp: "Formatted at 2026-09-25 13:07:29 on dist-test-slave-chkf"
server_key: "ede2dfb47971ac8ce8df595cc3331e95"
server_key_iv: "2611758ae84a0731b1abd445c8ee2447"
server_key_version: "encryptionkey@0"
I20260925 13:07:29.643224 5952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:29.667111 5952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:29.667464 5952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:29.667615 5952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:29.671408 5952 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:35327
I20260925 13:07:29.671446 6018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:35327 every 8 connection(s)
I20260925 13:07:29.671774 5952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:29.672673 6019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:29.672886 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 5952
I20260925 13:07:29.672992 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:07:29.673184 825 external_mini_cluster.cc:1428] Setting key c7c8f59e535b86a6c2f57376e91934bf
I20260925 13:07:29.676133 6019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5: Bootstrap starting.
I20260925 13:07:29.676720 6019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:29.677027 6019 log.cc:824] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:29.677739 6019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5: No bootstrap required, opened a new log
I20260925 13:07:29.679067 6022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.673683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40744 (local address 127.0.206.126:35327)
0925 13:07:29.673901 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:29.673910 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:29.673943 (+ 33us) server_negotiation.cc:407] Connection header received
0925 13:07:29.674389 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:29.674398 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:29.674480 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:29.674631 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:29.674815 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:29.676176 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:29.677024 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:29.677547 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:29.677667 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:29.678349 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:29.678352 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:29.678353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:07:29.678547 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:29.678587 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:07:29.678604 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:29.678675 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:29.678779 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 13:07:29.678851 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":76,"threads_started":1}
I20260925 13:07:29.680040 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:07:29.680169 6019 raft_consensus.cc:348] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17c6b2ee2fc4b45807635fc87613ad5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35327 } }
I20260925 13:07:29.680316 6019 raft_consensus.cc:374] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:29.680344 6019 raft_consensus.cc:729] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c17c6b2ee2fc4b45807635fc87613ad5, State: Initialized, Role: FOLLOWER
I20260925 13:07:29.680466 6019 consensus_queue.cc:260] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17c6b2ee2fc4b45807635fc87613ad5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35327 } }
I20260925 13:07:29.680542 6019 raft_consensus.cc:388] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:29.680581 6019 raft_consensus.cc:482] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:29.680624 6019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:29.692664 6019 raft_consensus.cc:504] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17c6b2ee2fc4b45807635fc87613ad5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35327 } }
I20260925 13:07:29.692802 6019 leader_election.cc:302] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c17c6b2ee2fc4b45807635fc87613ad5; no voters:
I20260925 13:07:29.692996 6019 leader_election.cc:288] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:07:29.693063 6024 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:29.693222 6024 raft_consensus.cc:686] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5 [term 1 LEADER]: Becoming Leader. State: Replica: c17c6b2ee2fc4b45807635fc87613ad5, State: Running, Role: LEADER
I20260925 13:07:29.693287 6019 sys_catalog.cc:564] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:29.693318 6024 consensus_queue.cc:237] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17c6b2ee2fc4b45807635fc87613ad5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35327 } }
I20260925 13:07:29.693634 6024 sys_catalog.cc:455] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c17c6b2ee2fc4b45807635fc87613ad5. Latest consensus state: current_term: 1 leader_uuid: "c17c6b2ee2fc4b45807635fc87613ad5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17c6b2ee2fc4b45807635fc87613ad5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35327 } } }
I20260925 13:07:29.693706 6024 sys_catalog.cc:458] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:29.693974 6025 sys_catalog.cc:455] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c17c6b2ee2fc4b45807635fc87613ad5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17c6b2ee2fc4b45807635fc87613ad5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35327 } } }
I20260925 13:07:29.694118 6025 sys_catalog.cc:458] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:29.694520 6029 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:29.695101 6029 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:07:29.696506 6029 catalog_manager.cc:1347] Generated new cluster ID: f838d943d9c545ec98b9ba8928f09a15
I20260925 13:07:29.696555 6029 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:07:29.709372 6029 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:07:29.709895 6029 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:29.716988 6029 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c17c6b2ee2fc4b45807635fc87613ad5: Generated new TSK 0
I20260925 13:07:29.717172 6029 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 13:07:29.745625 6049 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260925 13:07:29.745719 6049 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 13:07:29.745950 6022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.730645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40758 (local address 127.0.206.126:35327)
0925 13:07:29.730719 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:29.730721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:29.730834 (+ 113us) server_negotiation.cc:407] Connection header received
0925 13:07:29.730926 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:29.730927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:29.730941 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:29.731018 (+ 77us) server_negotiation.cc:226] Negotiated authn=JWT
0925 13:07:29.731145 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:29.731927 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:29.732619 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:29.732938 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:29.733000 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:29.733166 (+ 166us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 13:07:29.745727 (+ 12561us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 13:07:29.745871 (+ 144us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40758: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":34}
W20260925 13:07:29.746011 6022 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40758: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260925 13:07:29.746318 6048 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:29.730559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:35327 (local address 127.0.0.1:40758)
0925 13:07:29.730773 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:29.730782 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:29.730836 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:29.731025 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:29.731025 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:29.731027 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 13:07:29.731086 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:29.731088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:29.731997 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:29.731997 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:29.732549 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:29.732551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:29.733014 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:29.733014 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:29.733058 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:29.733074 (+ 16us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 13:07:29.745917 (+ 12843us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 13:07:29.746031 (+ 114us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.206.126:35327: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.0.206.126:35327: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260925 13:07:29.746811 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 5952
2026-09-25T13:07:29Z chronyd exiting
I20260925 13:07:29.782459 825 test_util.cc:175] -----------------------------------------------
I20260925 13:07:29.782548 825 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790341627249256-825-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (243 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260925 13:07:29.783320 825 mini_oidc.cc:152] Starting JWKS server
I20260925 13:07:29.783999 825 webserver.cc:460] Webserver started at https://127.0.0.1:34343/ using document root <none> and password file <none>
I20260925 13:07:29.784062 825 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 13:07:29.784227 825 webserver.cc:460] Webserver started at http://127.0.0.1:36225/ using document root <none> and password file <none>
2026-09-25T13:07:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:29Z Disabled control of system clock
I20260925 13:07:29.791965 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:34707
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:33621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:34707
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34343/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/testchainca.cert with env {}
W20260925 13:07:29.869990 6055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:29.870191 6055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:29.870213 6055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:29.871440 6055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:29.871483 6055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:29.871496 6055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:29.871508 6055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:29.871555 6055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 13:07:29.871572 6055 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 13:07:29.871594 6055 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34343/jwks.json
I20260925 13:07:29.873366 6055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:34707
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:34707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34343/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:29.873621 6055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:29.873836 6055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:29.878729 6063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:29.878742 6060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:29.878777 6061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:29.878960 6055 server_base.cc:1034] running on GCE node
I20260925 13:07:29.879170 6055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:29.879438 6055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:29.880579 6055 hybrid_clock.cc:648] HybridClock initialized: now 1790341649880567 us; error 28 us; skew 500 ppm
I20260925 13:07:29.881826 6055 webserver.cc:460] Webserver started at http://127.0.206.126:33449/ using document root <none> and password file <none>
I20260925 13:07:29.882064 6055 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:29.882112 6055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:29.882267 6055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:29.883376 6055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "1f7878d02e2043e49b7751a0dac21b50"
format_stamp: "Formatted at 2026-09-25 13:07:29 on dist-test-slave-chkf"
server_key: "fb563272919f0d37fa7ab90aa9b2663b"
server_key_iv: "bae2f7440b6021dae2590be494d50896"
server_key_version: "encryptionkey@0"
I20260925 13:07:29.883756 6055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "1f7878d02e2043e49b7751a0dac21b50"
format_stamp: "Formatted at 2026-09-25 13:07:29 on dist-test-slave-chkf"
server_key: "fb563272919f0d37fa7ab90aa9b2663b"
server_key_iv: "bae2f7440b6021dae2590be494d50896"
server_key_version: "encryptionkey@0"
I20260925 13:07:29.885051 6055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:07:29.885799 6069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:29.885977 6055 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:29.886065 6055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "1f7878d02e2043e49b7751a0dac21b50"
format_stamp: "Formatted at 2026-09-25 13:07:29 on dist-test-slave-chkf"
server_key: "fb563272919f0d37fa7ab90aa9b2663b"
server_key_iv: "bae2f7440b6021dae2590be494d50896"
server_key_version: "encryptionkey@0"
I20260925 13:07:29.886137 6055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:29.897039 6055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:29.897310 6055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:29.897449 6055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:29.901353 6055 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:34707
I20260925 13:07:29.901396 6121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:34707 every 8 connection(s)
I20260925 13:07:29.901743 6055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:29.902663 6122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:29.904830 6122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50: Bootstrap starting.
I20260925 13:07:29.905547 6122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:29.905602 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 6055
I20260925 13:07:29.905704 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:07:29.905933 825 external_mini_cluster.cc:1428] Setting key d17c1858bbb5271dd050932083984c11
I20260925 13:07:29.905913 6122 log.cc:824] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:29.906898 6122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50: No bootstrap required, opened a new log
I20260925 13:07:29.909027 6122 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f7878d02e2043e49b7751a0dac21b50" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 34707 } }
I20260925 13:07:29.909188 6122 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:29.909219 6122 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f7878d02e2043e49b7751a0dac21b50, State: Initialized, Role: FOLLOWER
I20260925 13:07:29.909325 6122 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f7878d02e2043e49b7751a0dac21b50" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 34707 } }
I20260925 13:07:29.909394 6122 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:29.909443 6122 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:29.909492 6122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:29.911437 6126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:29.906414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35148 (local address 127.0.206.126:34707)
0925 13:07:29.906607 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:29.906614 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:29.906639 (+ 25us) server_negotiation.cc:407] Connection header received
0925 13:07:29.907167 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:29.907174 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:29.907236 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:29.907349 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:29.907574 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:29.908437 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:29.909556 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:29.910156 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:29.910246 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:29.910871 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:29.910874 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:29.910875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:07:29.911030 (+ 155us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:29.911065 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:07:29.911075 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:29.911125 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:29.911179 (+ 54us) server_negotiation.cc:299] Negotiation successful
0925 13:07:29.911263 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
I20260925 13:07:29.912252 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:07:29.995222 6122 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f7878d02e2043e49b7751a0dac21b50" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 34707 } }
I20260925 13:07:29.995386 6122 leader_election.cc:302] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f7878d02e2043e49b7751a0dac21b50; no voters:
I20260925 13:07:29.995661 6122 leader_election.cc:288] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:07:29.995724 6127 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:29.995889 6127 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50 [term 1 LEADER]: Becoming Leader. State: Replica: 1f7878d02e2043e49b7751a0dac21b50, State: Running, Role: LEADER
I20260925 13:07:29.996012 6122 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:29.995998 6127 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f7878d02e2043e49b7751a0dac21b50" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 34707 } }
I20260925 13:07:29.996383 6129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f7878d02e2043e49b7751a0dac21b50. Latest consensus state: current_term: 1 leader_uuid: "1f7878d02e2043e49b7751a0dac21b50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f7878d02e2043e49b7751a0dac21b50" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 34707 } } }
I20260925 13:07:29.996394 6128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f7878d02e2043e49b7751a0dac21b50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f7878d02e2043e49b7751a0dac21b50" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 34707 } } }
I20260925 13:07:29.996582 6128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:29.996803 6129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:29.996971 6135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:29.997810 6135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:07:29.999588 6135 catalog_manager.cc:1347] Generated new cluster ID: fef49b8fc05348f3a6f5165a0ff7b824
I20260925 13:07:29.999641 6135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:07:30.040937 6135 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:07:30.041498 6135 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:30.044939 6135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f7878d02e2043e49b7751a0dac21b50: Generated new TSK 0
I20260925 13:07:30.045117 6135 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 13:07:30.067870 6152 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260925 13:07:30.067960 6152 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 13:07:30.068252 6126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:30.053120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35164 (local address 127.0.206.126:34707)
0925 13:07:30.053182 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:30.053184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:30.053311 (+ 127us) server_negotiation.cc:407] Connection header received
0925 13:07:30.053394 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:30.053395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:30.053405 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:30.053463 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0925 13:07:30.053605 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:30.054073 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:30.055133 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:30.055313 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:30.055363 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:30.055537 (+ 174us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 13:07:30.068081 (+ 12544us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34343/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 13:07:30.068202 (+ 121us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35164: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34343/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":36}
W20260925 13:07:30.068316 6126 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35164: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34343/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260925 13:07:30.068554 6151 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:30.053076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:34707 (local address 127.0.0.1:35164)
0925 13:07:30.053239 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:30.053251 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:30.053320 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:30.053479 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:30.053480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:30.053482 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 13:07:30.053546 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:30.053548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:30.054151 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:30.054151 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:30.055056 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:30.055059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:30.055383 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:30.055384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:30.055447 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:30.055453 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 13:07:30.068208 (+ 12755us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34343/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 13:07:30.068353 (+ 145us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.206.126:34707: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34343/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":112,"threads_started":1}
I20260925 13:07:30.068850 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 6055
2026-09-25T13:07:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (323 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260925 13:07:30.106968 825 mini_oidc.cc:152] Starting JWKS server
I20260925 13:07:30.107708 825 webserver.cc:460] Webserver started at https://127.0.0.1:42461/ using document root <none> and password file <none>
I20260925 13:07:30.107793 825 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 13:07:30.107959 825 webserver.cc:460] Webserver started at http://127.0.0.1:39147/ using document root <none> and password file <none>
2026-09-25T13:07:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:30Z Disabled control of system clock
I20260925 13:07:30.115854 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:42917
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:41703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:42917
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42461/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 13:07:30.194908 6158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:30.195093 6158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:30.195113 6158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:30.196201 6158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:30.196236 6158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:30.196249 6158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:30.196261 6158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:30.196273 6158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 13:07:30.196285 6158 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 13:07:30.196300 6158 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42461/jwks.json
I20260925 13:07:30.197556 6158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:42917
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:42917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42461/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:30.197731 6158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:30.197948 6158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:30.202580 6166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:30.202664 6158 server_base.cc:1034] running on GCE node
W20260925 13:07:30.202643 6164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:30.202803 6163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:30.203092 6158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:30.203378 6158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:30.204515 6158 hybrid_clock.cc:648] HybridClock initialized: now 1790341650204503 us; error 29 us; skew 500 ppm
I20260925 13:07:30.205657 6158 webserver.cc:460] Webserver started at http://127.0.206.126:33681/ using document root <none> and password file <none>
I20260925 13:07:30.205852 6158 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:30.205894 6158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:30.205979 6158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:30.207015 6158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "70c1af34899444deb3d46b5222947791"
format_stamp: "Formatted at 2026-09-25 13:07:30 on dist-test-slave-chkf"
server_key: "40686f600b66a3ec33e29e80d03cc4bb"
server_key_iv: "7ab1e9b9e97f77e1c9ac19c6dfa3c1a5"
server_key_version: "encryptionkey@0"
I20260925 13:07:30.207300 6158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "70c1af34899444deb3d46b5222947791"
format_stamp: "Formatted at 2026-09-25 13:07:30 on dist-test-slave-chkf"
server_key: "40686f600b66a3ec33e29e80d03cc4bb"
server_key_iv: "7ab1e9b9e97f77e1c9ac19c6dfa3c1a5"
server_key_version: "encryptionkey@0"
I20260925 13:07:30.208601 6158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:07:30.209223 6172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:30.209408 6158 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:30.209520 6158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "70c1af34899444deb3d46b5222947791"
format_stamp: "Formatted at 2026-09-25 13:07:30 on dist-test-slave-chkf"
server_key: "40686f600b66a3ec33e29e80d03cc4bb"
server_key_iv: "7ab1e9b9e97f77e1c9ac19c6dfa3c1a5"
server_key_version: "encryptionkey@0"
I20260925 13:07:30.209587 6158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:30.230885 6158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:30.231190 6158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:30.231335 6158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:30.235148 6158 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:42917
I20260925 13:07:30.235181 6224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:42917 every 8 connection(s)
I20260925 13:07:30.235536 6158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:30.236167 6225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:30.238999 6225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791: Bootstrap starting.
I20260925 13:07:30.239547 6225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:30.239898 6225 log.cc:824] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:30.239925 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 6158
I20260925 13:07:30.240018 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:07:30.240219 825 external_mini_cluster.cc:1428] Setting key 6a42454a214c89c619c8b4aafa16ee91
I20260925 13:07:30.240682 6225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791: No bootstrap required, opened a new log
I20260925 13:07:30.242429 6225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70c1af34899444deb3d46b5222947791" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42917 } }
I20260925 13:07:30.242576 6225 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:30.242621 6225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70c1af34899444deb3d46b5222947791, State: Initialized, Role: FOLLOWER
I20260925 13:07:30.242817 6225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70c1af34899444deb3d46b5222947791" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42917 } }
I20260925 13:07:30.242914 6225 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:30.242964 6225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:30.243012 6225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:30.246251 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:30.240657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46554 (local address 127.0.206.126:42917)
0925 13:07:30.240871 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:30.240880 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:30.240912 (+ 32us) server_negotiation.cc:407] Connection header received
0925 13:07:30.241799 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:30.241808 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:30.241872 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:30.242006 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:30.242201 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:30.243629 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:30.244432 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:30.244837 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:30.244967 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:30.245597 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:30.245601 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:30.245601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:07:30.245790 (+ 189us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:30.245831 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:07:30.245841 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:30.245904 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:30.246011 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 13:07:30.246090 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260925 13:07:30.247149 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:07:30.300125 6225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70c1af34899444deb3d46b5222947791" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42917 } }
I20260925 13:07:30.300310 6225 leader_election.cc:302] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70c1af34899444deb3d46b5222947791; no voters:
I20260925 13:07:30.300554 6225 leader_election.cc:288] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:07:30.300647 6230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:30.300808 6230 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791 [term 1 LEADER]: Becoming Leader. State: Replica: 70c1af34899444deb3d46b5222947791, State: Running, Role: LEADER
I20260925 13:07:30.300825 6225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:30.300922 6230 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70c1af34899444deb3d46b5222947791" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42917 } }
I20260925 13:07:30.301373 6231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70c1af34899444deb3d46b5222947791" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70c1af34899444deb3d46b5222947791" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42917 } } }
I20260925 13:07:30.301453 6231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:30.301643 6232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70c1af34899444deb3d46b5222947791. Latest consensus state: current_term: 1 leader_uuid: "70c1af34899444deb3d46b5222947791" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70c1af34899444deb3d46b5222947791" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42917 } } }
I20260925 13:07:30.301754 6232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:30.302220 6239 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:30.302893 6239 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:07:30.304855 6239 catalog_manager.cc:1347] Generated new cluster ID: ebd8b81607b84c6ca9ff2bc205786f47
I20260925 13:07:30.304924 6239 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:07:30.318341 6239 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:07:30.318887 6239 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:30.321578 6239 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70c1af34899444deb3d46b5222947791: Generated new TSK 0
I20260925 13:07:30.321739 6239 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:07:30.337698 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:30.334126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46566 (local address 127.0.206.126:42917)
0925 13:07:30.334172 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:30.334174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:30.334256 (+ 82us) server_negotiation.cc:407] Connection header received
0925 13:07:30.334322 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:30.334323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:30.334332 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:30.334389 (+ 57us) server_negotiation.cc:226] Negotiated authn=JWT
0925 13:07:30.334513 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:30.335044 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:30.336148 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:30.336416 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:30.336471 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:30.336656 (+ 185us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 13:07:30.337512 (+ 856us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42461/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 13:07:30.337634 (+ 122us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46566: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42461/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":24}
W20260925 13:07:30.337760 6229 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46566: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42461/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260925 13:07:30.337968 6254 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:30.334032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:42917 (local address 127.0.0.1:46566)
0925 13:07:30.334190 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:30.334201 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:30.334266 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:30.334404 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:30.334404 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:30.334407 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0925 13:07:30.334462 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:30.334464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:30.335113 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:30.335114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:30.336064 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:30.336067 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:30.336491 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:30.336492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:30.336555 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:30.336562 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 13:07:30.337607 (+ 1045us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42461/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 13:07:30.337703 (+ 96us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.206.126:42917: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42461/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
I20260925 13:07:30.338176 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 6158
E20260925 13:07:30.349066 6255 webserver.cc:533] Webserver: sslize failed:
E20260925 13:07:30.349146 6255 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-25T13:07:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (268 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-25T13:07:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:30Z Disabled control of system clock
I20260925 13:07:30.382695 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43431
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:45425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:43431
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/testchainca.cert with env {}
W20260925 13:07:30.464412 6259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:30.464648 6259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:30.464686 6259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:30.465844 6259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:30.465903 6259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:30.465924 6259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:30.465945 6259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:30.465973 6259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 13:07:30.465997 6259 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 13:07:30.466027 6259 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260925 13:07:30.467501 6259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:43431
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:30.467768 6259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:30.467993 6259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:30.472458 6267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:30.472446 6265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:30.472460 6264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:30.472729 6259 server_base.cc:1034] running on GCE node
I20260925 13:07:30.472991 6259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:30.473278 6259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:30.474431 6259 hybrid_clock.cc:648] HybridClock initialized: now 1790341650474416 us; error 28 us; skew 500 ppm
I20260925 13:07:30.475724 6259 webserver.cc:460] Webserver started at http://127.0.206.126:40125/ using document root <none> and password file <none>
I20260925 13:07:30.475953 6259 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:30.475998 6259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:30.476131 6259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:30.477124 6259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "55b228af49b74df1b371aa4238823492"
format_stamp: "Formatted at 2026-09-25 13:07:30 on dist-test-slave-chkf"
server_key: "2a54fe4f0bab5bd90d61e20fea8c6356"
server_key_iv: "72429eb6c1bc9f5c8e77d721b30f4a86"
server_key_version: "encryptionkey@0"
I20260925 13:07:30.477470 6259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "55b228af49b74df1b371aa4238823492"
format_stamp: "Formatted at 2026-09-25 13:07:30 on dist-test-slave-chkf"
server_key: "2a54fe4f0bab5bd90d61e20fea8c6356"
server_key_iv: "72429eb6c1bc9f5c8e77d721b30f4a86"
server_key_version: "encryptionkey@0"
I20260925 13:07:30.478670 6259 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:07:30.479375 6273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:30.479538 6259 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:30.479629 6259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "55b228af49b74df1b371aa4238823492"
format_stamp: "Formatted at 2026-09-25 13:07:30 on dist-test-slave-chkf"
server_key: "2a54fe4f0bab5bd90d61e20fea8c6356"
server_key_iv: "72429eb6c1bc9f5c8e77d721b30f4a86"
server_key_version: "encryptionkey@0"
I20260925 13:07:30.479707 6259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:30.495025 6259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:30.495332 6259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:30.495471 6259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:30.499521 6259 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:43431
I20260925 13:07:30.499572 6325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:43431 every 8 connection(s)
I20260925 13:07:30.499891 6259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:30.500771 6326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:30.503000 6326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492: Bootstrap starting.
I20260925 13:07:30.503568 6326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:30.503892 6326 log.cc:824] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:30.504731 6326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492: No bootstrap required, opened a new log
I20260925 13:07:30.506573 6326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b228af49b74df1b371aa4238823492" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43431 } }
I20260925 13:07:30.506682 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 6259
I20260925 13:07:30.506754 6326 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:30.506803 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:07:30.506803 6326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55b228af49b74df1b371aa4238823492, State: Initialized, Role: FOLLOWER
I20260925 13:07:30.506928 6326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b228af49b74df1b371aa4238823492" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43431 } }
I20260925 13:07:30.507014 825 external_mini_cluster.cc:1428] Setting key 007ed465218171f3274bc825c0a6497c
I20260925 13:07:30.507021 6326 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:30.507067 6326 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:30.507114 6326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:30.509797 6326 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b228af49b74df1b371aa4238823492" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43431 } }
I20260925 13:07:30.509908 6326 leader_election.cc:302] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55b228af49b74df1b371aa4238823492; no voters:
I20260925 13:07:30.510061 6326 leader_election.cc:288] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:07:30.510221 6331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:30.510361 6331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492 [term 1 LEADER]: Becoming Leader. State: Replica: 55b228af49b74df1b371aa4238823492, State: Running, Role: LEADER
I20260925 13:07:30.510413 6326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:30.510509 6331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b228af49b74df1b371aa4238823492" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43431 } }
I20260925 13:07:30.510878 6331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55b228af49b74df1b371aa4238823492" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b228af49b74df1b371aa4238823492" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43431 } } }
I20260925 13:07:30.511037 6331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:30.511220 6331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55b228af49b74df1b371aa4238823492. Latest consensus state: current_term: 1 leader_uuid: "55b228af49b74df1b371aa4238823492" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b228af49b74df1b371aa4238823492" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43431 } } }
I20260925 13:07:30.511318 6331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:30.511477 6340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:30.512051 6340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:07:30.512367 6330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:30.507484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45900 (local address 127.0.206.126:43431)
0925 13:07:30.507688 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:30.507700 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:30.507732 (+ 32us) server_negotiation.cc:407] Connection header received
0925 13:07:30.507826 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:30.507831 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:30.507880 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:30.507985 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:30.508146 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:30.508941 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:30.509738 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:30.510409 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:30.511310 (+ 901us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:30.511469 (+ 159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:30.511473 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:30.511473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:07:30.511680 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:30.511717 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:07:30.511731 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:30.512027 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:30.512058 (+ 31us) server_negotiation.cc:299] Negotiation successful
0925 13:07:30.512133 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
I20260925 13:07:30.513341 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:07:30.513774 6340 catalog_manager.cc:1347] Generated new cluster ID: 80f3f8d221084ad4ae0b663c01a696cb
I20260925 13:07:30.513885 6340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:07:30.521142 6340 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:07:30.521624 6340 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:30.528865 6340 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55b228af49b74df1b371aa4238823492: Generated new TSK 0
I20260925 13:07:30.529098 6340 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:07:30.601205 6330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:30.541234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45910 (local address 127.0.206.126:43431)
0925 13:07:30.541276 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:30.541278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:30.541372 (+ 94us) server_negotiation.cc:407] Connection header received
0925 13:07:30.541441 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:30.541441 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:30.541451 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:30.541509 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0925 13:07:30.541657 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:30.542132 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:30.543250 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:30.543434 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:30.543478 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:30.543645 (+ 167us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 13:07:30.600999 (+ 57354us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 13:07:30.601149 (+ 150us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45910: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":23}
W20260925 13:07:30.601284 6330 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45910: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260925 13:07:30.601392 6355 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:30.541150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43431 (local address 127.0.0.1:45910)
0925 13:07:30.541301 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:30.541312 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:30.541376 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:30.541529 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:30.541530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:30.541532 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 13:07:30.541588 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:30.541590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:30.542217 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:30.542218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:30.543167 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:30.543170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:30.543504 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:30.543505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:30.543562 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:30.543567 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 13:07:30.601133 (+ 57566us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 13:07:30.601274 (+ 141us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.206.126:43431: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260925 13:07:30.601756 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 6259
2026-09-25T13:07:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (243 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (87 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (88 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:07:32 dist-test-slave-chkf krb5kdc[6367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:07:32 dist-test-slave-chkf krb5kdc[6367](info): set up 2 sockets
Sep 25 13:07:32 dist-test-slave-chkf krb5kdc[6367](info): commencing operation
krb5kdc: starting...
W20260925 13:07:33.527331 825 mini_kdc.cc:121] Time spent starting KDC: real 2.732s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:07:33 dist-test-slave-chkf krb5kdc[6367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:07:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:07:33.641757 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:45571
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:45571
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:33.718909 6380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:33.719089 6380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:33.719111 6380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:33.720235 6380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:33.720274 6380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:33.720288 6380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:33.720301 6380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:33.720361 6380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:07:33.721809 6380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:45571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:45571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:33.722047 6380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:33.722259 6380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:33.726912 6388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:33.726890 6386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:33.726892 6385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:33.727200 6380 server_base.cc:1034] running on GCE node
I20260925 13:07:33.727407 6380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:33.727622 6380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:33.728763 6380 hybrid_clock.cc:648] HybridClock initialized: now 1790341653728751 us; error 28 us; skew 500 ppm
Sep 25 13:07:33 dist-test-slave-chkf krb5kdc[6367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341653, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:33.737783 6380 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:07:33.738220 6380 webserver.cc:460] Webserver started at http://127.0.206.126:35437/ using document root <none> and password file <none>
I20260925 13:07:33.738392 6380 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:33.738431 6380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:33.738513 6380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:33.739625 6380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "1ca13f3947774d17b73610f5d9202d42"
format_stamp: "Formatted at 2026-09-25 13:07:33 on dist-test-slave-chkf"
server_key: "888012670e1d793bae531f40060f015c"
server_key_iv: "69c16a70599d028da925e1f05a62ed78"
server_key_version: "encryptionkey@0"
I20260925 13:07:33.740064 6380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "1ca13f3947774d17b73610f5d9202d42"
format_stamp: "Formatted at 2026-09-25 13:07:33 on dist-test-slave-chkf"
server_key: "888012670e1d793bae531f40060f015c"
server_key_iv: "69c16a70599d028da925e1f05a62ed78"
server_key_version: "encryptionkey@0"
I20260925 13:07:33.741338 6380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 13:07:33.742039 6395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:33.742214 6380 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:07:33.742314 6380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "1ca13f3947774d17b73610f5d9202d42"
format_stamp: "Formatted at 2026-09-25 13:07:33 on dist-test-slave-chkf"
server_key: "888012670e1d793bae531f40060f015c"
server_key_iv: "69c16a70599d028da925e1f05a62ed78"
server_key_version: "encryptionkey@0"
I20260925 13:07:33.742403 6380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:33.753618 6380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:33.755445 6380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:33.755607 6380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:33.759591 6380 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:45571
I20260925 13:07:33.759641 6447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:45571 every 8 connection(s)
I20260925 13:07:33.760170 6380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:33.760771 6448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:33.762939 6448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42: Bootstrap starting.
I20260925 13:07:33.763441 6448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:33.763669 6448 log.cc:824] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:33.764257 6448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42: No bootstrap required, opened a new log
I20260925 13:07:33.765681 6448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca13f3947774d17b73610f5d9202d42" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45571 } }
I20260925 13:07:33.765769 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 6380
I20260925 13:07:33.765805 6448 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:33.765828 6448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca13f3947774d17b73610f5d9202d42, State: Initialized, Role: FOLLOWER
I20260925 13:07:33.765870 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:07:33.765991 6448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca13f3947774d17b73610f5d9202d42" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45571 } }
I20260925 13:07:33.766067 6448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:33.766088 825 external_mini_cluster.cc:1428] Setting key a2aa384d243753118479356a2c252b76
I20260925 13:07:33.766088 6448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:33.766163 6448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:33.766873 6448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca13f3947774d17b73610f5d9202d42" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45571 } }
I20260925 13:07:33.766964 6448 leader_election.cc:302] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ca13f3947774d17b73610f5d9202d42; no voters:
I20260925 13:07:33.767144 6448 leader_election.cc:288] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:07:33.767190 6453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:33.767356 6453 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42 [term 1 LEADER]: Becoming Leader. State: Replica: 1ca13f3947774d17b73610f5d9202d42, State: Running, Role: LEADER
I20260925 13:07:33.767398 6448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:33.768100 6453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca13f3947774d17b73610f5d9202d42" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45571 } }
I20260925 13:07:33.768780 6455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ca13f3947774d17b73610f5d9202d42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca13f3947774d17b73610f5d9202d42" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45571 } } }
I20260925 13:07:33.769040 6455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:33.769016 6454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ca13f3947774d17b73610f5d9202d42. Latest consensus state: current_term: 1 leader_uuid: "1ca13f3947774d17b73610f5d9202d42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca13f3947774d17b73610f5d9202d42" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45571 } } }
I20260925 13:07:33.769322 6469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:33.769323 6454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:33.769788 6469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:07:33.772070 6469 catalog_manager.cc:1347] Generated new cluster ID: d34aad7402ae46ff989bd99a5afafa7f
I20260925 13:07:33.772126 6469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 13:07:33 dist-test-slave-chkf krb5kdc[6367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:33.776959 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:33.766889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34738 (local address 127.0.206.126:45571)
0925 13:07:33.767066 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:33.767074 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:33.767102 (+ 28us) server_negotiation.cc:407] Connection header received
0925 13:07:33.767217 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:33.767224 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:33.767280 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:33.767437 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:33.768414 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:33.769888 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:33.770818 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:33.771413 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:33.771527 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:33.773802 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:33.773807 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:33.773807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:33.773816 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:33.775457 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:33.775841 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:33.775843 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:33.775844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:33.775948 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:33.776167 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:33.776168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:33.776169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:33.776396 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:33.776573 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:33.776634 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 13:07:33.776763 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
I20260925 13:07:33.806133 6469 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:07:33.806874 6469 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:33.809919 6469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ca13f3947774d17b73610f5d9202d42: Generated new TSK 0
I20260925 13:07:33.810081 6469 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260925 13:07:33.838057 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45571
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:33.917629 6475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:33.917819 6475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:33.917840 6475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:33.918910 6475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:33.918946 6475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:33.918970 6475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:07:33.920305 6475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:33.920466 6475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:33.920619 6475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:33.925309 6480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:33.925304 6481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:33.925443 6475 server_base.cc:1034] running on GCE node
W20260925 13:07:33.925304 6483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:33.925770 6475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:33.926011 6475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:33.927183 6475 hybrid_clock.cc:648] HybridClock initialized: now 1790341653927180 us; error 35 us; skew 500 ppm
Sep 25 13:07:33 dist-test-slave-chkf krb5kdc[6367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341653, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:33.929555 6475 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:07:33.929948 6475 webserver.cc:460] Webserver started at http://127.0.206.65:34449/ using document root <none> and password file <none>
I20260925 13:07:33.930156 6475 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:33.930225 6475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:33.930346 6475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:33.931389 6475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "f71fb4bf7f504308b1f99b77f41e7f17"
format_stamp: "Formatted at 2026-09-25 13:07:33 on dist-test-slave-chkf"
server_key: "68a49c11d094955cb79628629f227bf2"
server_key_iv: "7f40e459b0050daf005a91c0dd65a93e"
server_key_version: "encryptionkey@0"
I20260925 13:07:33.931732 6475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "f71fb4bf7f504308b1f99b77f41e7f17"
format_stamp: "Formatted at 2026-09-25 13:07:33 on dist-test-slave-chkf"
server_key: "68a49c11d094955cb79628629f227bf2"
server_key_iv: "7f40e459b0050daf005a91c0dd65a93e"
server_key_version: "encryptionkey@0"
I20260925 13:07:33.932962 6475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 13:07:33.933756 6490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:33.933938 6475 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:07:33.934041 6475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "f71fb4bf7f504308b1f99b77f41e7f17"
format_stamp: "Formatted at 2026-09-25 13:07:33 on dist-test-slave-chkf"
server_key: "68a49c11d094955cb79628629f227bf2"
server_key_iv: "7f40e459b0050daf005a91c0dd65a93e"
server_key_version: "encryptionkey@0"
I20260925 13:07:33.934124 6475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:33.951606 6475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:33.953406 6475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:33.953572 6475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:33.953799 6475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:33.954107 6475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:33.954165 6475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:33.954200 6475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:33.954227 6475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:33.959699 6475 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:37773
I20260925 13:07:33.959753 6603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:37773 every 8 connection(s)
I20260925 13:07:33.960078 6475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:07:33.962406 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 6475
I20260925 13:07:33.962520 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:07:33.962716 825 external_mini_cluster.cc:1428] Setting key 428eb63bfabebf769dbc0248b50851d8
Sep 25 13:07:33 dist-test-slave-chkf krb5kdc[6367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341653, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:33.970408 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:33.960774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:35021 (local address 127.0.206.126:45571)
0925 13:07:33.960820 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:33.960823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:33.961045 (+ 222us) server_negotiation.cc:407] Connection header received
0925 13:07:33.961921 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:33.961922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:33.961934 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:33.961985 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:33.962575 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:33.963214 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:33.964648 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:33.964874 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:33.964955 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:33.967165 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:33.967166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:33.967167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:33.967176 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:33.969020 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:33.969434 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:33.969435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:33.969436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:33.969489 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:33.969742 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:33.969743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:33.969743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:33.969885 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:33.969966 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:33.970230 (+ 264us) server_negotiation.cc:299] Negotiation successful
0925 13:07:33.970309 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 13:07:33.970571 6606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:33.960763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45571 (local address 127.0.206.65:35021)
0925 13:07:33.960922 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:33.960955 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:33.961822 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:33.962036 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:33.962039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:33.962290 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:33.962469 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:33.962473 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:33.963284 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:33.963285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:33.964565 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:33.964568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:33.964973 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:33.964974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:33.965095 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:33.965574 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:33.965581 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:33.967059 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:33.969124 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:33.969131 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:33.969133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:33.969342 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:33.969562 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:33.969563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:33.969564 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:33.969671 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:33.969957 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:33.969963 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:33.970097 (+ 134us) client_negotiation.cc:769] Sending connection context
0925 13:07:33.970208 (+ 111us) client_negotiation.cc:240] Negotiation successful
0925 13:07:33.970353 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
I20260925 13:07:33.971084 6604 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45571
I20260925 13:07:33.971174 6604 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:33.971367 6604 heartbeater.cc:511] Master 127.0.206.126:45571 requested a full tablet report, sending...
I20260925 13:07:33.971879 6412 ts_manager.cc:194] Registered new tserver with Master: f71fb4bf7f504308b1f99b77f41e7f17 (127.0.206.65:37773)
I20260925 13:07:33.972492 6412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:35021
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260925 13:07:33.987653 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45571
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:34.066493 6609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:34.066676 6609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:34.066756 6609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:34.067947 6609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:34.067988 6609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:34.068015 6609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:07:34.069550 6609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:34.069777 6609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:34.070009 6609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:07:34.074962 6609 server_base.cc:1034] running on GCE node
W20260925 13:07:34.074914 6615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:34.074940 6614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:34.074914 6617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:34.075392 6609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:34.075624 6609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:34.076798 6609 hybrid_clock.cc:648] HybridClock initialized: now 1790341654076777 us; error 33 us; skew 500 ppm
Sep 25 13:07:34 dist-test-slave-chkf krb5kdc[6367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341654, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:34.079830 6609 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:07:34.080345 6609 webserver.cc:460] Webserver started at http://127.0.206.66:40459/ using document root <none> and password file <none>
I20260925 13:07:34.080587 6609 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:34.080632 6609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:34.080765 6609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:34.081743 6609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "3f5c81b632e34d46ac3829d9c678f08c"
format_stamp: "Formatted at 2026-09-25 13:07:34 on dist-test-slave-chkf"
server_key: "8ef23ec7233014fcb709ed0690c5ad27"
server_key_iv: "b9aaf787cd0f5b1cb7e5e3164bb98ab7"
server_key_version: "encryptionkey@0"
I20260925 13:07:34.082082 6609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "3f5c81b632e34d46ac3829d9c678f08c"
format_stamp: "Formatted at 2026-09-25 13:07:34 on dist-test-slave-chkf"
server_key: "8ef23ec7233014fcb709ed0690c5ad27"
server_key_iv: "b9aaf787cd0f5b1cb7e5e3164bb98ab7"
server_key_version: "encryptionkey@0"
I20260925 13:07:34.083438 6609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 13:07:34.084123 6624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:34.084321 6609 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:07:34.084386 6609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "3f5c81b632e34d46ac3829d9c678f08c"
format_stamp: "Formatted at 2026-09-25 13:07:34 on dist-test-slave-chkf"
server_key: "8ef23ec7233014fcb709ed0690c5ad27"
server_key_iv: "b9aaf787cd0f5b1cb7e5e3164bb98ab7"
server_key_version: "encryptionkey@0"
I20260925 13:07:34.084491 6609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:34.127200 6609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:34.129593 6609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:34.129747 6609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:34.130002 6609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:34.130373 6609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:34.130430 6609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:34.130474 6609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:34.130501 6609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:34.136410 6609 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:45591
I20260925 13:07:34.136471 6737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:45591 every 8 connection(s)
I20260925 13:07:34.136831 6609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:07:34.142221 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 6609
I20260925 13:07:34.142302 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:07:34.142498 825 external_mini_cluster.cc:1428] Setting key a4d814ed091a3ed69d23c72cbaef870d
Sep 25 13:07:34 dist-test-slave-chkf krb5kdc[6367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341654, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:34.146831 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:34.137627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:58089 (local address 127.0.206.126:45571)
0925 13:07:34.137716 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:34.137718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:34.137917 (+ 199us) server_negotiation.cc:407] Connection header received
0925 13:07:34.138912 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:34.138913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:34.138923 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:34.138982 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:34.139551 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:34.140035 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:34.141003 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:34.141177 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:34.141226 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:34.143551 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:34.143553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:34.143553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:34.143560 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:34.145364 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:34.145755 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:34.145756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:34.145756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:34.145818 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:34.146104 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:34.146105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:34.146105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:34.146250 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:34.146320 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:34.146559 (+ 239us) server_negotiation.cc:299] Negotiation successful
0925 13:07:34.146655 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 13:07:34.146848 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:34.137599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45571 (local address 127.0.206.66:58089)
0925 13:07:34.137784 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:34.137828 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:34.138806 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:34.139045 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:34.139048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:34.139295 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:34.139481 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:34.139485 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:34.140127 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:34.140128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:34.140926 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:34.140928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:34.141260 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:34.141261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:34.141365 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:34.141928 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:34.141936 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:34.143433 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:34.145448 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:34.145454 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:34.145455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:34.145664 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:34.145895 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:34.145923 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:34.145923 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:34.146030 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:34.146318 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:34.146322 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:34.146430 (+ 108us) client_negotiation.cc:769] Sending connection context
0925 13:07:34.146511 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 13:07:34.146622 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":105,"threads_started":1}
I20260925 13:07:34.147354 6738 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45571
I20260925 13:07:34.147445 6738 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:34.147624 6738 heartbeater.cc:511] Master 127.0.206.126:45571 requested a full tablet report, sending...
I20260925 13:07:34.148012 6412 ts_manager.cc:194] Registered new tserver with Master: 3f5c81b632e34d46ac3829d9c678f08c (127.0.206.66:45591)
I20260925 13:07:34.148386 6412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:58089
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260925 13:07:34.169955 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45571
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:34.247665 6743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:34.247929 6743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:34.247975 6743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:34.249102 6743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:34.249172 6743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:34.249217 6743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:07:34.250674 6743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:34.250950 6743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:34.251170 6743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:34.255926 6749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:34.255934 6748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:34.255952 6743 server_base.cc:1034] running on GCE node
W20260925 13:07:34.255950 6751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:34.256421 6743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:34.256649 6743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:34.257819 6743 hybrid_clock.cc:648] HybridClock initialized: now 1790341654257799 us; error 32 us; skew 500 ppm
Sep 25 13:07:34 dist-test-slave-chkf krb5kdc[6367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341654, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:34.260252 6743 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:07:34.260684 6743 webserver.cc:460] Webserver started at http://127.0.206.67:37077/ using document root <none> and password file <none>
I20260925 13:07:34.260851 6743 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:34.260890 6743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:34.260980 6743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:34.261878 6743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "2a1b561737884c5282b559438182d218"
format_stamp: "Formatted at 2026-09-25 13:07:34 on dist-test-slave-chkf"
server_key: "7989ed58f3221c6b0d5ca8c20d884808"
server_key_iv: "ca407e7d1c6a92f77dae9ddca42a9a89"
server_key_version: "encryptionkey@0"
I20260925 13:07:34.262172 6743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "2a1b561737884c5282b559438182d218"
format_stamp: "Formatted at 2026-09-25 13:07:34 on dist-test-slave-chkf"
server_key: "7989ed58f3221c6b0d5ca8c20d884808"
server_key_iv: "ca407e7d1c6a92f77dae9ddca42a9a89"
server_key_version: "encryptionkey@0"
I20260925 13:07:34.263438 6743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:07:34.264111 6758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:34.264305 6743 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:07:34.264415 6743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "2a1b561737884c5282b559438182d218"
format_stamp: "Formatted at 2026-09-25 13:07:34 on dist-test-slave-chkf"
server_key: "7989ed58f3221c6b0d5ca8c20d884808"
server_key_iv: "ca407e7d1c6a92f77dae9ddca42a9a89"
server_key_version: "encryptionkey@0"
I20260925 13:07:34.264484 6743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:34.272841 6743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:34.275060 6743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:34.275215 6743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:34.275444 6743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:34.275722 6743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:34.275753 6743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:34.275771 6743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:34.275784 6743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:34.281105 6743 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:42309
I20260925 13:07:34.281164 6871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:42309 every 8 connection(s)
I20260925 13:07:34.281445 6743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:07:34.284062 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 6743
I20260925 13:07:34.284127 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:07:34.284298 825 external_mini_cluster.cc:1428] Setting key 53a3c772d9083641277682e827a26222
Sep 25 13:07:34 dist-test-slave-chkf krb5kdc[6367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341654, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:34.290395 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:34.282143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:57131 (local address 127.0.206.126:45571)
0925 13:07:34.282199 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:34.282201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:34.282382 (+ 181us) server_negotiation.cc:407] Connection header received
0925 13:07:34.283319 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:34.283320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:34.283329 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:34.283385 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:34.283916 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:34.284478 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:34.285694 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:34.285882 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:34.285938 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:34.288058 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:34.288059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:34.288060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:34.288066 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:34.289342 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:34.289686 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:34.289687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:34.289687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:34.289736 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:34.289901 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:34.289902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:34.289902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:34.289997 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:34.290048 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:34.290268 (+ 220us) server_negotiation.cc:299] Negotiation successful
0925 13:07:34.290336 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 13:07:34.290524 6874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:34.282069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45571 (local address 127.0.206.67:57131)
0925 13:07:34.282257 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:34.282294 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:34.283226 (+ 932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:34.283455 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:34.283458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:34.283673 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:34.283830 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:34.283834 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:34.284612 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:34.284613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:34.285628 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:34.285630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:34.285961 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:34.285962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:34.286050 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:34.286540 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:34.286550 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:34.287962 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:34.289416 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:34.289421 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:34.289423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:34.289620 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:34.289775 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:34.289776 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:34.289776 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:34.289850 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:34.290058 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:34.290062 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:34.290160 (+ 98us) client_negotiation.cc:769] Sending connection context
0925 13:07:34.290237 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 13:07:34.290341 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":79,"threads_started":1}
I20260925 13:07:34.290951 6872 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45571
I20260925 13:07:34.291051 6872 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:34.291206 6872 heartbeater.cc:511] Master 127.0.206.126:45571 requested a full tablet report, sending...
I20260925 13:07:34.291544 6412 ts_manager.cc:194] Registered new tserver with Master: 2a1b561737884c5282b559438182d218 (127.0.206.67:42309)
I20260925 13:07:34.291863 6412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:57131
I20260925 13:07:34.295913 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/test-admin.keytab.
Sep 25 13:07:34 dist-test-slave-chkf krb5kdc[6367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:34.311237 825 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260925 13:07:34.312757 6879 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:34.312060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34798 (local address 127.0.206.65:37773)
0925 13:07:34.312205 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:34.312210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:34.312225 (+ 15us) server_negotiation.cc:407] Connection header received
0925 13:07:34.312248 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:34.312250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:34.312262 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:34.312327 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:34.312652 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34798: BlockingRecv error: recv got EOF from 127.0.0.1:34798 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
W20260925 13:07:34.312929 825 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/krb5cc' of type FILE
Sep 25 13:07:34 dist-test-slave-chkf krb5kdc[6367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:34.313876 825 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 13:07:34 dist-test-slave-chkf krb5kdc[6367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:34.314849 825 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 13:07:34 dist-test-slave-chkf krb5kdc[6367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260925 13:07:34.322646 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:34.315349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34814 (local address 127.0.206.65:37773)
0925 13:07:34.315421 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:34.315423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:34.315469 (+ 46us) server_negotiation.cc:407] Connection header received
0925 13:07:34.315524 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:34.315525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:34.315535 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:34.315584 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:34.315960 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:34.316620 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:34.317392 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:34.317703 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:34.317788 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:34.319871 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:34.319878 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:34.319878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:34.319885 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:34.321450 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:34.321852 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:34.321855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:34.321855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:34.321912 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:34.322131 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:34.322132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:34.322132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:34.322258 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:34.322413 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:34.322473 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 13:07:34.322540 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260925 13:07:34.324388 6879 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:34.323901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34816 (local address 127.0.206.65:37773)
0925 13:07:34.323935 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:34.323936 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:34.324018 (+ 82us) server_negotiation.cc:407] Connection header received
0925 13:07:34.324048 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:34.324048 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:34.324057 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:34.324106 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:34.324319 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34816: BlockingRecv error: recv got EOF from 127.0.0.1:34816 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 13:07:34.324483 825 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/krb5cc' of type FILE
Sep 25 13:07:34 dist-test-slave-chkf krb5kdc[6367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:34.325423 825 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 13:07:34 dist-test-slave-chkf krb5kdc[6367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:34.326259 825 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 13:07:34 dist-test-slave-chkf krb5kdc[6367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260925 13:07:34.332476 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:34.326758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34830 (local address 127.0.206.65:37773)
0925 13:07:34.326796 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:34.326798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:34.326872 (+ 74us) server_negotiation.cc:407] Connection header received
0925 13:07:34.326913 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:34.326914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:34.326921 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:34.326964 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:34.327261 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:34.327711 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:34.328353 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:34.328536 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:34.328583 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:34.330343 (+ 1760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:34.330344 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:34.330344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:34.330350 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:34.331690 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:34.331977 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:34.331978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:34.331978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:34.332023 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:34.332164 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:34.332165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:34.332165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:34.332233 (+ 68us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:34.332292 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:34.332374 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 13:07:34.332427 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 13:07:34.334002 6879 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:34.333460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34832 (local address 127.0.206.65:37773)
0925 13:07:34.333498 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:34.333500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:34.333541 (+ 41us) server_negotiation.cc:407] Connection header received
0925 13:07:34.333586 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:34.333586 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:34.333593 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:34.333634 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:34.333931 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34832: BlockingRecv error: recv got EOF from 127.0.0.1:34832 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260925 13:07:34.334090 825 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790341627249256-825-0/krb5kdc/krb5cc' of type FILE
Sep 25 13:07:34 dist-test-slave-chkf krb5kdc[6367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:34.335047 825 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 13:07:34 dist-test-slave-chkf krb5kdc[6367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:34.335922 825 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 13:07:34 dist-test-slave-chkf krb5kdc[6367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260925 13:07:34.342360 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:34.336448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34834 (local address 127.0.206.65:37773)
0925 13:07:34.336487 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:34.336489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:34.336536 (+ 47us) server_negotiation.cc:407] Connection header received
0925 13:07:34.336600 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:34.336601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:34.336611 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:34.336666 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:34.336982 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:34.337408 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:34.338081 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:34.338249 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:34.338300 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:34.340184 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:34.340185 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:34.340185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:34.340191 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:34.341501 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:34.341783 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:34.341784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:34.341784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:34.341829 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:34.341974 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:34.341974 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:34.341974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:34.342044 (+ 70us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:34.342102 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:34.342188 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 13:07:34.342235 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 13:07:34.342976 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 6475
I20260925 13:07:34.348469 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 6609
I20260925 13:07:34.352929 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 6743
I20260925 13:07:34.356928 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 6380
2026-09-25T13:07:34Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3581 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:07:34 dist-test-slave-chkf krb5kdc[6892](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:07:34 dist-test-slave-chkf krb5kdc[6892](info): set up 2 sockets
Sep 25 13:07:34 dist-test-slave-chkf krb5kdc[6892](info): commencing operation
krb5kdc: starting...
W20260925 13:07:35.382862 825 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:07:35 dist-test-slave-chkf krb5kdc[6892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:07:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:35Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.126.keytab.
Entry for principal oryx/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.126.keytab.
I20260925 13:07:35.505554 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38937
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.126.keytab
--principal=oryx/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:38937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:35.584501 6905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:35.584738 6905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:35.584777 6905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:35.585904 6905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:35.585981 6905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:35.586006 6905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:35.586035 6905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:35.586066 6905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:07:35.587519 6905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:38937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.126.keytab
--principal=oryx/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:35.587822 6905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:35.588057 6905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:35.593010 6910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:35.593104 6905 server_base.cc:1034] running on GCE node
W20260925 13:07:35.593175 6911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:35.593010 6913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:35.593488 6905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:35.593815 6905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:35.594965 6905 hybrid_clock.cc:648] HybridClock initialized: now 1790341655594952 us; error 49 us; skew 500 ppm
Sep 25 13:07:35 dist-test-slave-chkf krb5kdc[6892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341655, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:35.597429 6905 init.cc:377] Logged in from keytab as oryx/127.0.206.126@KRBTEST.COM (short username oryx)
I20260925 13:07:35.597890 6905 webserver.cc:460] Webserver started at http://127.0.206.126:40791/ using document root <none> and password file <none>
I20260925 13:07:35.598129 6905 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:35.598202 6905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:35.598366 6905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:35.599571 6905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "0b09bc81bfcd4976a0b4dc056fa50948"
format_stamp: "Formatted at 2026-09-25 13:07:35 on dist-test-slave-chkf"
server_key: "f4a016b0dab00acbc6cbd495ee5ac0f3"
server_key_iv: "6d768f71217dc8d3535486cd6aeb7f7c"
server_key_version: "encryptionkey@0"
I20260925 13:07:35.599927 6905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "0b09bc81bfcd4976a0b4dc056fa50948"
format_stamp: "Formatted at 2026-09-25 13:07:35 on dist-test-slave-chkf"
server_key: "f4a016b0dab00acbc6cbd495ee5ac0f3"
server_key_iv: "6d768f71217dc8d3535486cd6aeb7f7c"
server_key_version: "encryptionkey@0"
I20260925 13:07:35.601203 6905 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:07:35.601936 6920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:35.602219 6905 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:35.602312 6905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "0b09bc81bfcd4976a0b4dc056fa50948"
format_stamp: "Formatted at 2026-09-25 13:07:35 on dist-test-slave-chkf"
server_key: "f4a016b0dab00acbc6cbd495ee5ac0f3"
server_key_iv: "6d768f71217dc8d3535486cd6aeb7f7c"
server_key_version: "encryptionkey@0"
I20260925 13:07:35.602419 6905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:35.612325 6905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:35.614213 6905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:35.614387 6905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:35.618433 6905 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:38937
I20260925 13:07:35.618527 6972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:38937 every 8 connection(s)
I20260925 13:07:35.618913 6905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:35.619591 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 6905
I20260925 13:07:35.619732 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:07:35.619951 825 external_mini_cluster.cc:1428] Setting key de8a3c9af09a20e1ece1febfc470ead9
I20260925 13:07:35.619968 6973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:35.623039 6973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948: Bootstrap starting.
I20260925 13:07:35.623657 6973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:35.623965 6973 log.cc:824] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:35.624683 6973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948: No bootstrap required, opened a new log
I20260925 13:07:35.626678 6973 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } }
I20260925 13:07:35.626889 6973 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:35.626930 6973 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b09bc81bfcd4976a0b4dc056fa50948, State: Initialized, Role: FOLLOWER
I20260925 13:07:35.627049 6973 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } }
Sep 25 13:07:35 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260925 13:07:35.627125 6973 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:35.627161 6973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:35.627213 6973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:35.627879 6973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } }
I20260925 13:07:35.627996 6973 leader_election.cc:302] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b09bc81bfcd4976a0b4dc056fa50948; no voters:
I20260925 13:07:35.628190 6973 leader_election.cc:288] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:07:35.628250 6978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:35.628393 6978 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [term 1 LEADER]: Becoming Leader. State: Replica: 0b09bc81bfcd4976a0b4dc056fa50948, State: Running, Role: LEADER
I20260925 13:07:35.628466 6973 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:35.628484 6978 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } }
I20260925 13:07:35.629069 6979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } }
I20260925 13:07:35.629142 6979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:35.629288 6980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b09bc81bfcd4976a0b4dc056fa50948. Latest consensus state: current_term: 1 leader_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } }
I20260925 13:07:35.629341 6980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:35.629492 6987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:35.630000 6987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:07:35.631714 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:35.620496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34164 (local address 127.0.206.126:38937)
0925 13:07:35.620701 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:35.620709 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:35.620740 (+ 31us) server_negotiation.cc:407] Connection header received
0925 13:07:35.621331 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:35.621339 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:35.621386 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:35.621508 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:35.622307 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:35.623427 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:35.624503 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:35.625079 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:35.625192 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:35.627657 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:35.627660 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:35.627660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:35.627669 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:35.630002 (+ 2333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:35.630478 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:35.630480 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:35.630480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:35.630606 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:35.630823 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:35.630825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:35.630825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:35.631121 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:35.631263 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:35.631400 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 13:07:35.631530 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":108,"threads_started":1}
I20260925 13:07:35.635110 6987 catalog_manager.cc:1347] Generated new cluster ID: d4ab679f95aa42aaadb7f7d3c98d2d00
I20260925 13:07:35.635165 6987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:07:35.664767 6987 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:07:35.665280 6987 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:35.682561 6987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948: Generated new TSK 0
I20260925 13:07:35.682775 6987 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.65.keytab.
Entry for principal oryx/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.65.keytab.
I20260925 13:07:35.714017 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38937
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.65.keytab
--principal=oryx/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:35.796563 6999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:35.796777 6999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:35.796814 6999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:35.797930 6999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:35.798009 6999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:35.798056 6999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:07:35.799602 6999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.65.keytab
--principal=oryx/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:35.799853 6999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:35.800082 6999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:35.805014 7004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:35.805162 7007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:35.805174 7005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:35.805250 6999 server_base.cc:1034] running on GCE node
I20260925 13:07:35.805497 6999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:35.805737 6999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:35.806912 6999 hybrid_clock.cc:648] HybridClock initialized: now 1790341655806898 us; error 24 us; skew 500 ppm
Sep 25 13:07:35 dist-test-slave-chkf krb5kdc[6892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341655, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:35.809347 6999 init.cc:377] Logged in from keytab as oryx/127.0.206.65@KRBTEST.COM (short username oryx)
I20260925 13:07:35.809698 6999 webserver.cc:460] Webserver started at http://127.0.206.65:43307/ using document root <none> and password file <none>
I20260925 13:07:35.809891 6999 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:35.809929 6999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:35.810015 6999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:35.811058 6999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a"
format_stamp: "Formatted at 2026-09-25 13:07:35 on dist-test-slave-chkf"
server_key: "3123f7d770dff503d443bccaabe149b9"
server_key_iv: "3b37c43d7364cb755e79b8795429330c"
server_key_version: "encryptionkey@0"
I20260925 13:07:35.811350 6999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a"
format_stamp: "Formatted at 2026-09-25 13:07:35 on dist-test-slave-chkf"
server_key: "3123f7d770dff503d443bccaabe149b9"
server_key_iv: "3b37c43d7364cb755e79b8795429330c"
server_key_version: "encryptionkey@0"
I20260925 13:07:35.813061 6999 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 13:07:35.813810 7014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:35.814020 6999 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:07:35.814093 6999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a"
format_stamp: "Formatted at 2026-09-25 13:07:35 on dist-test-slave-chkf"
server_key: "3123f7d770dff503d443bccaabe149b9"
server_key_iv: "3b37c43d7364cb755e79b8795429330c"
server_key_version: "encryptionkey@0"
I20260925 13:07:35.814185 6999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:35.826630 6999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:35.828563 6999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:35.828725 6999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:35.828979 6999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:35.829324 6999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:35.829383 6999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:35.829422 6999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:35.829449 6999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:35.835040 6999 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:45033
I20260925 13:07:35.835125 7127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:45033 every 8 connection(s)
I20260925 13:07:35.835417 6999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:07:35.838891 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 6999
I20260925 13:07:35.838979 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:07:35.839216 825 external_mini_cluster.cc:1428] Setting key 1b09ddfd5af5df29fe6996e081cb6393
Sep 25 13:07:35 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341655, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260925 13:07:35.845093 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:35.836139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:40567 (local address 127.0.206.126:38937)
0925 13:07:35.836209 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:35.836211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:35.836412 (+ 201us) server_negotiation.cc:407] Connection header received
0925 13:07:35.836826 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:35.836827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:35.836837 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:35.836887 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:35.837721 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:35.838228 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:35.839365 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:35.839644 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:35.839734 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:35.842230 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:35.842231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:35.842232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:35.842239 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:35.843711 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:35.844138 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:35.844139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:35.844140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:35.844195 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:35.844464 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:35.844465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:35.844466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:35.844581 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:35.844663 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:35.844923 (+ 260us) server_negotiation.cc:299] Negotiation successful
0925 13:07:35.845004 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 13:07:35.845273 7130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:35.836143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.65:40567)
0925 13:07:35.836299 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:35.836329 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:35.836742 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:35.836945 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:35.836948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:35.837448 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:35.837611 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:35.837615 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:35.838313 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:35.838314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:35.839294 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:35.839297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:35.839742 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:35.839744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:35.839873 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:35.840367 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:35.840377 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:35.842129 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:35.843818 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:35.843825 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:35.843827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:35.844044 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:35.844282 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:35.844283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:35.844283 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:35.844389 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:35.844654 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:35.844659 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:35.844791 (+ 132us) client_negotiation.cc:769] Sending connection context
0925 13:07:35.844903 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 13:07:35.845052 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260925 13:07:35.845913 7128 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38937
I20260925 13:07:35.846020 7128 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:35.846190 7128 heartbeater.cc:511] Master 127.0.206.126:38937 requested a full tablet report, sending...
I20260925 13:07:35.846799 6937 ts_manager.cc:194] Registered new tserver with Master: 5fc6c6780d8f4d14b8fe491aa3bc687a (127.0.206.65:45033)
I20260925 13:07:35.847690 6937 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:40567
WARNING: no policy specified for oryx/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.66.keytab.
Entry for principal oryx/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.66.keytab.
I20260925 13:07:35.865993 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38937
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.66.keytab
--principal=oryx/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:35.947270 7133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:35.947446 7133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:35.947469 7133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:35.948634 7133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:35.948678 7133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:35.948702 7133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:07:35.950222 7133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.66.keytab
--principal=oryx/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:35.950464 7133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:35.950696 7133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:35.955513 7138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:35.955502 7139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:35.955502 7141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:35.955859 7133 server_base.cc:1034] running on GCE node
I20260925 13:07:35.956056 7133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:35.956239 7133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:35.957373 7133 hybrid_clock.cc:648] HybridClock initialized: now 1790341655957352 us; error 37 us; skew 500 ppm
Sep 25 13:07:35 dist-test-slave-chkf krb5kdc[6892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341655, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:35.959797 7133 init.cc:377] Logged in from keytab as oryx/127.0.206.66@KRBTEST.COM (short username oryx)
I20260925 13:07:35.960350 7133 webserver.cc:460] Webserver started at http://127.0.206.66:39195/ using document root <none> and password file <none>
I20260925 13:07:35.960618 7133 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:35.960695 7133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:35.960824 7133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:35.961822 7133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d"
format_stamp: "Formatted at 2026-09-25 13:07:35 on dist-test-slave-chkf"
server_key: "aa89670db4d688e4aca1d9e95dde4507"
server_key_iv: "2e115c3e2cb3dca81a669a3c0d00f1ea"
server_key_version: "encryptionkey@0"
I20260925 13:07:35.962199 7133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d"
format_stamp: "Formatted at 2026-09-25 13:07:35 on dist-test-slave-chkf"
server_key: "aa89670db4d688e4aca1d9e95dde4507"
server_key_iv: "2e115c3e2cb3dca81a669a3c0d00f1ea"
server_key_version: "encryptionkey@0"
I20260925 13:07:35.963558 7133 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:07:35.964231 7148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:35.964411 7133 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:35.964514 7133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d"
format_stamp: "Formatted at 2026-09-25 13:07:35 on dist-test-slave-chkf"
server_key: "aa89670db4d688e4aca1d9e95dde4507"
server_key_iv: "2e115c3e2cb3dca81a669a3c0d00f1ea"
server_key_version: "encryptionkey@0"
I20260925 13:07:35.964597 7133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:35.989429 7133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:35.991288 7133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:35.991456 7133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:35.991671 7133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:35.991981 7133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:35.992041 7133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:35.992076 7133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:35.992102 7133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:35.997707 7133 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:32855
I20260925 13:07:35.997756 7261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:32855 every 8 connection(s)
I20260925 13:07:35.998049 7133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:07:36.000872 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 7133
I20260925 13:07:36.001003 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:07:36.001204 825 external_mini_cluster.cc:1428] Setting key 80a34d279efca2ce868bf3c377f46f2d
Sep 25 13:07:35 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341655, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260925 13:07:36.008385 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:35.998770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:34491 (local address 127.0.206.126:38937)
0925 13:07:35.998851 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:35.998853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:35.999060 (+ 207us) server_negotiation.cc:407] Connection header received
0925 13:07:35.999486 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:35.999487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:35.999497 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:35.999546 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.000517 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.001049 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.002473 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.002657 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.002772 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.005185 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.005187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.005187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.005195 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.007029 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.007418 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.007420 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.007420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.007483 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.007734 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.007735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.007736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.007875 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.007951 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.008193 (+ 242us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.008278 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 13:07:36.008486 7264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:35.998748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.66:34491)
0925 13:07:35.998934 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:35.998967 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:35.999398 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:35.999604 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:35.999607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.000248 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.000430 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.000435 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.001130 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.001131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.002389 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.002392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.002770 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.002771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.002923 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.003391 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.003402 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.005096 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.007123 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.007129 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.007130 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.007337 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.007558 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.007559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.007559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.007647 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.007948 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.007952 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.008053 (+ 101us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.008134 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.008240 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":105,"threads_started":1}
I20260925 13:07:36.008963 7262 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38937
I20260925 13:07:36.009074 7262 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:36.009232 7262 heartbeater.cc:511] Master 127.0.206.126:38937 requested a full tablet report, sending...
I20260925 13:07:36.009639 6937 ts_manager.cc:194] Registered new tserver with Master: a1ea26f9b21b4a16af8b2434fd43cc7d (127.0.206.66:32855)
I20260925 13:07:36.010052 6937 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:34491
WARNING: no policy specified for oryx/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.67.keytab.
Entry for principal oryx/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.67.keytab.
I20260925 13:07:36.030095 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38937
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.67.keytab
--principal=oryx/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:36.109123 7267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:36.109337 7267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:36.109375 7267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:36.110489 7267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:36.110561 7267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:36.110605 7267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:07:36.112062 7267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.67.keytab
--principal=oryx/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:36.112308 7267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:36.112545 7267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:36.117408 7273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:36.117524 7267 server_base.cc:1034] running on GCE node
W20260925 13:07:36.117398 7275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:36.117399 7272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:36.117820 7267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:36.118093 7267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:36.119261 7267 hybrid_clock.cc:648] HybridClock initialized: now 1790341656119245 us; error 26 us; skew 500 ppm
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:36.121562 7267 init.cc:377] Logged in from keytab as oryx/127.0.206.67@KRBTEST.COM (short username oryx)
I20260925 13:07:36.122016 7267 webserver.cc:460] Webserver started at http://127.0.206.67:35567/ using document root <none> and password file <none>
I20260925 13:07:36.122185 7267 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:36.122223 7267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:36.122308 7267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:36.123307 7267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "79e6f0ab9db24450a82431ce46059fda"
format_stamp: "Formatted at 2026-09-25 13:07:36 on dist-test-slave-chkf"
server_key: "762953716b1a67d053aef2e51533bb44"
server_key_iv: "a5af32e401fa284bb21d02a162eadc96"
server_key_version: "encryptionkey@0"
I20260925 13:07:36.123592 7267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "79e6f0ab9db24450a82431ce46059fda"
format_stamp: "Formatted at 2026-09-25 13:07:36 on dist-test-slave-chkf"
server_key: "762953716b1a67d053aef2e51533bb44"
server_key_iv: "a5af32e401fa284bb21d02a162eadc96"
server_key_version: "encryptionkey@0"
I20260925 13:07:36.124830 7267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:07:36.125509 7282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:36.125710 7267 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:36.125779 7267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "79e6f0ab9db24450a82431ce46059fda"
format_stamp: "Formatted at 2026-09-25 13:07:36 on dist-test-slave-chkf"
server_key: "762953716b1a67d053aef2e51533bb44"
server_key_iv: "a5af32e401fa284bb21d02a162eadc96"
server_key_version: "encryptionkey@0"
I20260925 13:07:36.125876 7267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:36.153148 7267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:36.155211 7267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:36.155356 7267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:36.155606 7267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:36.155922 7267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:36.155962 7267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:36.155985 7267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:36.156031 7267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:36.161937 7267 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:37123
I20260925 13:07:36.161994 7395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:37123 every 8 connection(s)
I20260925 13:07:36.162339 7267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:07:36.164703 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 7267
I20260925 13:07:36.164788 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:07:36.165007 825 external_mini_cluster.cc:1428] Setting key 5c03795b41304dfa7984d8cf3f19916e
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260925 13:07:36.172206 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.163121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:43547 (local address 127.0.206.126:38937)
0925 13:07:36.163199 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.163202 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.163387 (+ 185us) server_negotiation.cc:407] Connection header received
0925 13:07:36.163921 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.163922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.163936 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.163995 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.164879 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.165630 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.166988 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.167226 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.167277 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.169554 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.169555 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.169556 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.169564 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.170862 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.171267 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.171268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.171268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.171326 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.171583 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.171584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.171585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.171693 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.171777 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.172043 (+ 266us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.172118 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 13:07:36.172382 7398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.163092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.67:43547)
0925 13:07:36.163259 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.163299 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.163752 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.164044 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.164047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.164571 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.164785 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.164791 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.165790 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.165791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.166909 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.166912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.167307 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.167308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.167416 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.168006 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.168014 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.169458 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.170949 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.170955 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.170957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.171166 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.171405 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.171406 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.171406 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.171507 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.171767 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.171773 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.171904 (+ 131us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.172021 (+ 117us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.172170 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260925 13:07:36.172910 7396 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38937
I20260925 13:07:36.173002 7396 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:36.173137 7396 heartbeater.cc:511] Master 127.0.206.126:38937 requested a full tablet report, sending...
I20260925 13:07:36.173491 6937 ts_manager.cc:194] Registered new tserver with Master: 79e6f0ab9db24450a82431ce46059fda (127.0.206.67:37123)
I20260925 13:07:36.173861 6937 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:43547
I20260925 13:07:36.176718 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.206.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.125.keytab.
Entry for principal oryx/127.0.206.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.125.keytab.
I20260925 13:07:36.202795 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.125:43157
--webserver_interface=127.0.206.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.125.keytab
--principal=oryx/127.0.206.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:38937,127.0.206.125:43157
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:36.281435 7401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:36.281613 7401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:36.281651 7401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:36.282774 7401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:36.282817 7401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:36.282842 7401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:36.282857 7401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:36.282869 7401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:07:36.284237 7401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:38937,127.0.206.125:43157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.125.keytab
--principal=oryx/127.0.206.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.125:43157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.206.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
W20260925 13:07:36.284328 7401 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.0.206.126:38937,127.0.206.125:43157'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260925 13:07:36.284447 7401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:36.284593 7401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:36.289251 7406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:36.289256 7407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:36.289366 7409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:36.289489 7401 server_base.cc:1034] running on GCE node
I20260925 13:07:36.289798 7401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:36.290031 7401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:36.291198 7401 hybrid_clock.cc:648] HybridClock initialized: now 1790341656291183 us; error 48 us; skew 500 ppm
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:36.293430 7401 init.cc:377] Logged in from keytab as oryx/127.0.206.125@KRBTEST.COM (short username oryx)
I20260925 13:07:36.293875 7401 webserver.cc:460] Webserver started at http://127.0.206.125:36965/ using document root <none> and password file <none>
I20260925 13:07:36.294108 7401 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:36.294190 7401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:36.294333 7401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:36.295360 7401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/data/instance:
uuid: "0ca60ab3dcb2456395272077367c4e44"
format_stamp: "Formatted at 2026-09-25 13:07:36 on dist-test-slave-chkf"
server_key: "39e5825c15fcb995b0e72a603f17188d"
server_key_iv: "890527f83f8579282a1de07c0cd70d95"
server_key_version: "encryptionkey@0"
I20260925 13:07:36.295708 7401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/wal/instance:
uuid: "0ca60ab3dcb2456395272077367c4e44"
format_stamp: "Formatted at 2026-09-25 13:07:36 on dist-test-slave-chkf"
server_key: "39e5825c15fcb995b0e72a603f17188d"
server_key_iv: "890527f83f8579282a1de07c0cd70d95"
server_key_version: "encryptionkey@0"
I20260925 13:07:36.296965 7401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:07:36.297648 7416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:36.297830 7401 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:36.297937 7401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/wal
uuid: "0ca60ab3dcb2456395272077367c4e44"
format_stamp: "Formatted at 2026-09-25 13:07:36 on dist-test-slave-chkf"
server_key: "39e5825c15fcb995b0e72a603f17188d"
server_key_iv: "890527f83f8579282a1de07c0cd70d95"
server_key_version: "encryptionkey@0"
I20260925 13:07:36.298019 7401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:36.310000 7401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:36.311829 7401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:36.311996 7401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:36.315996 7401 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.125:43157
I20260925 13:07:36.316025 7468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.125:43157 every 8 connection(s)
I20260925 13:07:36.316349 7401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/data/info.pb
I20260925 13:07:36.316951 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 7401
I20260925 13:07:36.317507 7469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:36.319784 7469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.206.125@KRBTEST.COM
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.125@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260925 13:07:36.327453 7472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.317589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34722 (local address 127.0.206.125:43157)
0925 13:07:36.317789 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.317797 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.317859 (+ 62us) server_negotiation.cc:407] Connection header received
0925 13:07:36.318314 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.318319 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.318360 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.318469 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.319236 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.320131 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.321104 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.321530 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.321607 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.324396 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.324402 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.324402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.324408 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.325898 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.326268 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.326271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.326271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.326348 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.326567 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.326568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.326568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.326853 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.327013 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.327120 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.327237 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
I20260925 13:07:36.327818 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.320344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:34168)
0925 13:07:36.320529 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.320557 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.320608 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.320771 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.320773 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.320993 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.321050 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.321051 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.321788 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.321789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.322409 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.322410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.322951 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.322952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.323010 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.323410 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.323419 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.324915 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.326609 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.326615 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.326617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.326887 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.327119 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.327120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.327120 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.327177 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.327487 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.327493 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.327541 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.327624 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.327739 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":91,"threads_started":1}
I20260925 13:07:36.328240 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.320414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34168 (local address 127.0.206.126:38937)
0925 13:07:36.320474 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.320476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.320628 (+ 152us) server_negotiation.cc:407] Connection header received
0925 13:07:36.320675 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.320676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.320687 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.320749 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.321169 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.321698 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.322505 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.322871 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.322942 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.324997 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.324999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.324999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.325006 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.326504 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.326984 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.326985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.326986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.327046 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.327254 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.327255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.327255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.327404 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.327496 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.327677 (+ 181us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.328154 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 13:07:36.328766 7469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.125@KRBTEST.COM for oryx/127.0.206.125@KRBTEST.COM
I20260925 13:07:36.334836 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.328974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43157 (local address 127.0.0.1:34724)
0925 13:07:36.329028 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.329038 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.329075 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.329222 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.329223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.329349 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.329371 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.329372 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.330131 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.330132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.330686 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.330687 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.331089 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.331090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.331149 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.331474 (+ 325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.331478 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.332503 (+ 1025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.333807 (+ 1304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.333808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.333808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.333967 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.334174 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.334175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.334175 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.334246 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.334522 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.334522 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.334557 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.334624 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.334734 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 13:07:36.334885 7472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.329034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34724 (local address 127.0.206.125:43157)
0925 13:07:36.329064 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.329066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.329074 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:36.329130 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.329131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.329139 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.329221 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.329508 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.330027 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.330805 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.331002 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.331089 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.332601 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.332602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.332603 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.332609 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.333706 (+ 1097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.334054 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.334055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.334056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.334106 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.334355 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.334356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.334356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.334451 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.334528 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.334665 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.334781 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":16}
I20260925 13:07:36.336031 7469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: Bootstrap starting.
I20260925 13:07:36.336802 7469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:36.337086 7469 log.cc:824] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:36.337718 7469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: No bootstrap required, opened a new log
I20260925 13:07:36.339123 7469 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } }
I20260925 13:07:36.339232 7469 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:36.339254 7469 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ca60ab3dcb2456395272077367c4e44, State: Initialized, Role: FOLLOWER
I20260925 13:07:36.339397 7469 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } }
I20260925 13:07:36.339643 7475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } } }
I20260925 13:07:36.339681 7469 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:36.339711 7475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 13:07:36.341472 7486 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:07:36.341523 7486 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:07:36.342976 7401 master_runner.cc:428] Detected that this master 0ca60ab3dcb2456395272077367c4e44 is joining an existing cluster
I20260925 13:07:36.343014 7401 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.206.125:43157
I20260925 13:07:36.350950 7492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.343903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:34170)
0925 13:07:36.344037 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.344047 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.344102 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.344291 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.344292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.344510 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.344590 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.344591 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.345252 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.345253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.345920 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.345926 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.346335 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.346335 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.346395 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.346887 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.346892 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.347223 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.348967 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.348968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.348968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.349205 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.350160 (+ 955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.350161 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.350161 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.350256 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.350576 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.350577 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.350626 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.350757 (+ 131us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.350847 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":19,"thread_start_us":81,"threads_started":1}
I20260925 13:07:36.350944 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.344007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34170 (local address 127.0.206.126:38937)
0925 13:07:36.344049 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.344051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.344089 (+ 38us) server_negotiation.cc:407] Connection header received
0925 13:07:36.344186 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.344187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.344199 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.344230 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.344675 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.345153 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.346027 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.346234 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.346323 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.347317 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.347319 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.347319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.347326 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.348870 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.349288 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.349289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.349289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.349349 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.350337 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.350338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.350339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.350478 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.350527 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.350776 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.350855 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 13:07:36.351150 7472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.344152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34736 (local address 127.0.206.125:43157)
0925 13:07:36.344194 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.344196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.344346 (+ 150us) server_negotiation.cc:407] Connection header received
0925 13:07:36.344416 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.344417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.344432 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.344479 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.344848 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.345553 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.346575 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.346866 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.346904 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.348379 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.348381 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.348381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.348387 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.350068 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.350414 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.350415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.350415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.350463 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.350660 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.350661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.350661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.350791 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.350852 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.351015 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.351079 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 13:07:36.351439 7493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.344148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43157 (local address 127.0.0.1:34736)
0925 13:07:36.344251 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.344262 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.344349 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.344497 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.344497 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.344700 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.344767 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.344768 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.345646 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.345646 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.346486 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.346488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.346966 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.346966 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.347020 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.347592 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.347597 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.348289 (+ 692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.350147 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.350147 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.350148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.350314 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.350534 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.350534 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.350535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.350585 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.350873 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.350873 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.350907 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.351242 (+ 335us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.351322 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":1,"thread_start_us":66,"threads_started":1}
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341655, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.126@KRBTEST.COM for oryx/127.0.206.125@KRBTEST.COM
I20260925 13:07:36.360064 7472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.353607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34742 (local address 127.0.206.125:43157)
0925 13:07:36.353669 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.353671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.353679 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:36.353748 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.353749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.353758 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.353801 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.354196 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.354621 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.355410 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.355607 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.355696 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.357507 (+ 1811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.357509 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.357509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.357515 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.358873 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.359260 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.359261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.359261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.359310 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.359530 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.359530 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.359531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.359635 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.359713 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.359897 (+ 184us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.359975 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 13:07:36.360085 7494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.353470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43157 (local address 127.0.0.1:34742)
0925 13:07:36.353564 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.353590 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.353673 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.353814 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.353818 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.354068 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.354109 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.354110 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.354732 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.354733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.355304 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.355306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.355699 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.355701 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.355779 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.356184 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.356188 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.357409 (+ 1221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.358970 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.358976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.358978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.359168 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.359390 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.359391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.359392 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.359462 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.359705 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.359711 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.359758 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.359871 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.359979 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":57,"threads_started":1}
I20260925 13:07:36.360770 6937 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.206.125:43157: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0ca60ab3dcb2456395272077367c4e44"
member_type: NON_VOTER
last_known_addr {
host: "127.0.206.125"
port: 43157
}
attrs {
promote: true
}
}
dest_uuid: "0b09bc81bfcd4976a0b4dc056fa50948"
cas_config_opid_index: -1
I20260925 13:07:36.361078 6937 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: true } }
I20260925 13:07:36.361538 7496 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0ca60ab3dcb2456395272077367c4e44 (127.0.206.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: true } } }
I20260925 13:07:36.361879 7443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:36.362586 7496 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.206.125:43157
I20260925 13:07:36.362666 7496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: true } } }
I20260925 13:07:36.362771 7496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:36.362810 7443 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 FOLLOWER]: Refusing update from remote peer 0b09bc81bfcd4976a0b4dc056fa50948: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260925 13:07:36.362879 7401 master.cc:537] Master@127.0.206.125:43157 shutting down...
I20260925 13:07:36.363256 7495 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 13:07:36.364078 6924 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260925 13:07:36.365200 6924 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 -> Peer 0ca60ab3dcb2456395272077367c4e44 (127.0.206.125:43157): Couldn't send request to peer 0ca60ab3dcb2456395272077367c4e44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:07:36.365537 7401 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:36.365617 7401 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:36.365682 7401 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: stopping tablet replica
I20260925 13:07:36.377960 7401 master.cc:552] Master@127.0.206.125:43157 shutdown complete.
I20260925 13:07:36.377998 7401 master_runner.cc:305] Clearing existing system tablet
I20260925 13:07:36.378150 7401 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:36.378213 7401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:36.379001 7401 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:36.379504 7500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:36.379626 7401 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:36.379681 7401 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:36.379703 7401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/wal
uuid: "0ca60ab3dcb2456395272077367c4e44"
format_stamp: "Formatted at 2026-09-25 13:07:36 on dist-test-slave-chkf"
server_key: "39e5825c15fcb995b0e72a603f17188d"
server_key_iv: "890527f83f8579282a1de07c0cd70d95"
server_key_version: "encryptionkey@0"
I20260925 13:07:36.380646 7401 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:36.382232 7401 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 13:07:36.382300 7401 log.cc:1196] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260925 13:07:36.382514 7401 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: Deleting consensus metadata
I20260925 13:07:36.382644 7401 master_runner.cc:315] Copying system tablet from 127.0.206.126:38937
I20260925 13:07:36.383252 7401 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: tablet copy: Beginning tablet copy session from remote peer at address 127.0.206.126:38937
I20260925 13:07:36.389220 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.383621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34184 (local address 127.0.206.126:38937)
0925 13:07:36.383666 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.383668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.383727 (+ 59us) server_negotiation.cc:407] Connection header received
0925 13:07:36.383803 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.383804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.383813 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.383868 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.384186 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.384677 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.385438 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.385650 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.385741 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.386666 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.386667 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.386668 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.386674 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.388047 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.388415 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.388416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.388416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.388482 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.388686 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.388687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.388687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.388800 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.388880 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.389056 (+ 176us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.389129 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 13:07:36.389235 7506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.383558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:34184)
0925 13:07:36.383668 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.383679 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.383743 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.383873 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.383874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.384047 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.384099 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.384101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.384765 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.384766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.385338 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.385340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.385744 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.385745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.385812 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.386291 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.386295 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.386557 (+ 262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.388150 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.388151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.388151 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.388324 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.388553 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.388554 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.388554 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.388610 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.388870 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.388871 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.388916 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.389027 (+ 111us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.389129 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":63,"threads_started":1}
I20260925 13:07:36.389653 6957 tablet_copy_service.cc:140] P 0b09bc81bfcd4976a0b4dc056fa50948: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0ca60ab3dcb2456395272077367c4e44 ({username='oryx', principal='oryx/127.0.206.125@KRBTEST.COM'} at 127.0.0.1:34184)
I20260925 13:07:36.389771 6957 tablet_copy_service.cc:161] P 0b09bc81bfcd4976a0b4dc056fa50948: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0ca60ab3dcb2456395272077367c4e44 at {username='oryx', principal='oryx/127.0.206.125@KRBTEST.COM'} at 127.0.0.1:34184: session id = 0ca60ab3dcb2456395272077367c4e44-00000000000000000000000000000000
I20260925 13:07:36.390538 6957 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 13:07:36.391155 7401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 13:07:36.391995 7401 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: tablet copy: Starting download of 0 data blocks...
I20260925 13:07:36.392155 7401 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: tablet copy: Starting download of 1 WAL segments...
I20260925 13:07:36.393031 7401 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 13:07:36.393780 6957 tablet_copy_service.cc:342] P 0b09bc81bfcd4976a0b4dc056fa50948: Request end of tablet copy session 0ca60ab3dcb2456395272077367c4e44-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.206.125@KRBTEST.COM'} at 127.0.0.1:34184
I20260925 13:07:36.393848 6957 tablet_copy_service.cc:433] P 0b09bc81bfcd4976a0b4dc056fa50948: ending tablet copy session 0ca60ab3dcb2456395272077367c4e44-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0ca60ab3dcb2456395272077367c4e44
I20260925 13:07:36.396468 7401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:36.397624 7507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:36.397707 7508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:36.397823 7401 server_base.cc:1034] running on GCE node
W20260925 13:07:36.397959 7510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:36.398089 7401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:36.398204 7401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:36.399317 7401 hybrid_clock.cc:648] HybridClock initialized: now 1790341656399307 us; error 25 us; skew 500 ppm
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:36.400503 7401 init.cc:377] Logged in from keytab as oryx/127.0.206.125@KRBTEST.COM (short username oryx)
I20260925 13:07:36.400763 7401 webserver.cc:460] Webserver started at http://127.0.206.125:41533/ using document root <none> and password file <none>
I20260925 13:07:36.400851 7401 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:36.400874 7401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:36.401417 7401 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:36.401876 7517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:36.402004 7401 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:36.402050 7401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/wal
uuid: "0ca60ab3dcb2456395272077367c4e44"
format_stamp: "Formatted at 2026-09-25 13:07:36 on dist-test-slave-chkf"
server_key: "39e5825c15fcb995b0e72a603f17188d"
server_key_iv: "890527f83f8579282a1de07c0cd70d95"
server_key_version: "encryptionkey@0"
I20260925 13:07:36.402097 7401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:36.422768 7401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:36.424345 7401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:36.427402 7401 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.125:43157
I20260925 13:07:36.427416 7569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.125:43157 every 8 connection(s)
I20260925 13:07:36.427740 7401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-1/data/info.pb
I20260925 13:07:36.428596 7570 sys_catalog.cc:263] Verifying existing consensus state
I20260925 13:07:36.429153 7570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: Bootstrap starting.
I20260925 13:07:36.431985 7570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:07:36.432515 7570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: Bootstrap complete.
I20260925 13:07:36.432716 7570 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: true } }
I20260925 13:07:36.432801 7570 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0ca60ab3dcb2456395272077367c4e44, State: Initialized, Role: LEARNER
I20260925 13:07:36.432840 7570 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: true } }
I20260925 13:07:36.432991 7570 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:36.434193 7574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: true } } }
I20260925 13:07:36.434389 7574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [sys.catalog]: This master's current role is: LEARNER
I20260925 13:07:36.434784 7585 catalog_manager.cc:1259] Loaded cluster ID: d4ab679f95aa42aaadb7f7d3c98d2d00
I20260925 13:07:36.434829 7585 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: loading cluster ID for follower catalog manager: success
I20260925 13:07:36.436261 7585 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: acquiring CA information for follower catalog manager: success
I20260925 13:07:36.436574 7585 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 13:07:36.437251 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.429171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34758 (local address 127.0.206.125:43157)
0925 13:07:36.429279 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.429281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.429288 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:07:36.429309 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.429310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.429319 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.429459 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.430113 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.430993 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.431696 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.431916 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.432015 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.433332 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.433333 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.433334 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.433340 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.435906 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.436380 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.436382 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.436382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.436441 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.436686 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.436687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.436688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.436816 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.436917 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.437050 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.437143 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for oryx/127.0.206.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.124.keytab.
Entry for principal oryx/127.0.206.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.124.keytab.
I20260925 13:07:36.462198 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.124:36989
--webserver_interface=127.0.206.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.124.keytab
--principal=oryx/127.0.206.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:38937,127.0.206.125:43157,127.0.206.124:36989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:36.542909 7588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:36.543090 7588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:36.543112 7588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:36.544166 7588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:36.544200 7588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:36.544214 7588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:36.544226 7588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:36.544238 7588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:07:36.545610 7588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:38937,127.0.206.125:43157,127.0.206.124:36989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.124.keytab
--principal=oryx/127.0.206.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.124:36989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.206.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:36.545791 7588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:36.545953 7588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:36.550922 7593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:36.550990 7588 server_base.cc:1034] running on GCE node
W20260925 13:07:36.550916 7596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:36.550917 7594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:36.551335 7588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:36.551565 7588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:36.552754 7588 hybrid_clock.cc:648] HybridClock initialized: now 1790341656552742 us; error 49 us; skew 500 ppm
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:36.554975 7588 init.cc:377] Logged in from keytab as oryx/127.0.206.124@KRBTEST.COM (short username oryx)
I20260925 13:07:36.555434 7588 webserver.cc:460] Webserver started at http://127.0.206.124:38467/ using document root <none> and password file <none>
I20260925 13:07:36.555675 7588 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:36.555747 7588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:36.555903 7588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:36.556870 7588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/data/instance:
uuid: "8895dce2e73c44d9be634ce437bfbaab"
format_stamp: "Formatted at 2026-09-25 13:07:36 on dist-test-slave-chkf"
server_key: "4ea41f4e8b7605cb7f7085bda3d07944"
server_key_iv: "9b26ed86ce06375f1121b994662c1f14"
server_key_version: "encryptionkey@0"
I20260925 13:07:36.557228 7588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/wal/instance:
uuid: "8895dce2e73c44d9be634ce437bfbaab"
format_stamp: "Formatted at 2026-09-25 13:07:36 on dist-test-slave-chkf"
server_key: "4ea41f4e8b7605cb7f7085bda3d07944"
server_key_iv: "9b26ed86ce06375f1121b994662c1f14"
server_key_version: "encryptionkey@0"
I20260925 13:07:36.558506 7588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:07:36.559307 7603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:36.559476 7588 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:36.559587 7588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/wal
uuid: "8895dce2e73c44d9be634ce437bfbaab"
format_stamp: "Formatted at 2026-09-25 13:07:36 on dist-test-slave-chkf"
server_key: "4ea41f4e8b7605cb7f7085bda3d07944"
server_key_iv: "9b26ed86ce06375f1121b994662c1f14"
server_key_version: "encryptionkey@0"
I20260925 13:07:36.559674 7588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:36.590281 7588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:36.592231 7588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:36.592398 7588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:36.597251 7588 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.124:36989
I20260925 13:07:36.597335 7655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.124:36989 every 8 connection(s)
I20260925 13:07:36.597690 7588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/data/info.pb
I20260925 13:07:36.598599 7656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:36.600131 7656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.124@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260925 13:07:36.606928 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 7588
I20260925 13:07:36.609364 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.600885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34200 (local address 127.0.206.126:38937)
0925 13:07:36.600941 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.600943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.601123 (+ 180us) server_negotiation.cc:407] Connection header received
0925 13:07:36.601604 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.601605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.601622 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.601679 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.602366 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.602932 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.603984 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.604184 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.604277 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.606428 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.606429 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.606430 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.606437 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.608055 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.608421 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.608422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.608423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.608486 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.608726 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.608727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.608727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.608849 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.608933 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.609201 (+ 268us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.609276 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 13:07:36.609551 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.600715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:34200)
0925 13:07:36.600915 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.601001 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.601491 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.601747 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.601751 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.602077 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.602256 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.602261 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.603068 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.603069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.603872 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.603874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.604307 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.604308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.604428 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.604850 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.604858 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.606325 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.608130 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.608134 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.608135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.608341 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.608542 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.608543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.608544 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.608647 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.608925 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.608930 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.609059 (+ 129us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.609167 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.609325 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":57,"threads_started":1}
I20260925 13:07:36.610009 7656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.206.124@KRBTEST.COM
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.124@KRBTEST.COM for oryx/127.0.206.125@KRBTEST.COM
I20260925 13:07:36.616655 7659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.607490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34660 (local address 127.0.206.124:36989)
0925 13:07:36.607625 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.607629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.607646 (+ 17us) server_negotiation.cc:407] Connection header received
0925 13:07:36.607673 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.607675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.607686 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.607765 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.608482 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.609120 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.610153 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.610617 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.610702 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.613162 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.613168 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.613168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.613173 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.615421 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.615798 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.615800 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.615801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.615854 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.616067 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.616068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.616068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.616252 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.616419 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.616465 (+ 46us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.616543 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":85,"threads_started":1}
I20260925 13:07:36.617112 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.610222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43157 (local address 127.0.0.1:34766)
0925 13:07:36.610345 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.610353 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.610385 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.610515 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.610516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.610617 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.610655 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.610656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.611383 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.611384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.611879 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.611881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.612373 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.612374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.612428 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.613172 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.613176 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.614339 (+ 1163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.615996 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.615996 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.615997 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.616201 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.616418 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.616419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.616419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.616479 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.616873 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.616874 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.616914 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.616986 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.617060 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260925 13:07:36.617285 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.610313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34766 (local address 127.0.206.125:43157)
0925 13:07:36.610363 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.610365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.610377 (+ 12us) server_negotiation.cc:407] Connection header received
0925 13:07:36.610442 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.610443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.610454 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.610507 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.610809 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.611276 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.611984 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.612287 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.612356 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.614435 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.614437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.614437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.614444 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.615915 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.616266 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.616267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.616267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.616343 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.616557 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.616557 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.616558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.616686 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.616759 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.617035 (+ 276us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.617120 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 13:07:36.617595 7656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.124@KRBTEST.COM for oryx/127.0.206.124@KRBTEST.COM
I20260925 13:07:36.623375 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.617827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:36989 (local address 127.0.0.1:34674)
0925 13:07:36.617878 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.617888 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.617924 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.618029 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.618030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.618154 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.618174 (+ 20us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.618175 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.618833 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.618834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.619287 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.619289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.619681 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.619682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.619729 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.620145 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.620149 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.621186 (+ 1037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.622455 (+ 1269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.622455 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.622456 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.622607 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.622867 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.622868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.622868 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.622924 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.623142 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.623143 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.623173 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.623223 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.623290 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 13:07:36.623420 7659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.617898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34674 (local address 127.0.206.124:36989)
0925 13:07:36.617930 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.617932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.617947 (+ 15us) server_negotiation.cc:407] Connection header received
0925 13:07:36.617964 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.617965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.617973 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.618014 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.618251 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.618698 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.619391 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.619595 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.619682 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.621269 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.621270 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.621270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.621275 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.622382 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.622681 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.622682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.622682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.622780 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.622986 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.622987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.622987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.623075 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.623130 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.623239 (+ 109us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.623332 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":19}
I20260925 13:07:36.624662 7656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: Bootstrap starting.
I20260925 13:07:36.625456 7656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:36.625736 7656 log.cc:824] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:36.626444 7656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: No bootstrap required, opened a new log
I20260925 13:07:36.628082 7656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } }
I20260925 13:07:36.628255 7656 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:36.628283 7656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8895dce2e73c44d9be634ce437bfbaab, State: Initialized, Role: FOLLOWER
I20260925 13:07:36.628515 7656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } }
I20260925 13:07:36.628894 7656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:36.628799 7661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } } }
I20260925 13:07:36.628997 7661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [sys.catalog]: This master's current role is: FOLLOWER
W20260925 13:07:36.630767 7672 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:07:36.630924 7672 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:07:36.633026 7588 master_runner.cc:428] Detected that this master 8895dce2e73c44d9be634ce437bfbaab is joining an existing cluster
I20260925 13:07:36.633062 7588 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.206.124:36989
I20260925 13:07:36.641212 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.634284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34768 (local address 127.0.206.125:43157)
0925 13:07:36.634318 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.634320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.634825 (+ 505us) server_negotiation.cc:407] Connection header received
0925 13:07:36.634850 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.634851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.634863 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.634927 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.635314 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.636004 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.637461 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.637670 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.637726 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.638677 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.638678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.638679 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.638684 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.640147 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.640536 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.640537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.640537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.640583 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.640782 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.640783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.640783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.640882 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.640932 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.641100 (+ 168us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.641152 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:07:36.641395 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.634232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43157 (local address 127.0.0.1:34768)
0925 13:07:36.634659 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.634667 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.634759 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.634952 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.634952 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.635135 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.635224 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.635226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.636092 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.636093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.637324 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.637327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.637752 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.637753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.637816 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.638192 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.638197 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.638591 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.640234 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.640236 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.640236 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.640463 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.640656 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.640656 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.640657 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.640720 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.640954 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.640955 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.641008 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.641084 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.641174 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":93,"threads_started":1}
I20260925 13:07:36.646837 7659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.634498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34690 (local address 127.0.206.124:36989)
0925 13:07:36.634531 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.634532 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.641521 (+ 6989us) server_negotiation.cc:407] Connection header received
0925 13:07:36.641600 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.641601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.641608 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.641639 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.641961 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.642381 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.643362 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.643528 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.643579 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.644431 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.644431 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.644432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.644437 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.645784 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.646143 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.646144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.646144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.646188 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.646376 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.646376 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.646377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.646457 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.646502 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.646649 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.646744 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:07:36.646965 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.634560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:36989 (local address 127.0.0.1:34690)
0925 13:07:36.641472 (+ 6912us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.641482 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.641540 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.641684 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.641684 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.641826 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.641876 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.641878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.642469 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.642470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.643284 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.643287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.643603 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.643604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.643669 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.644002 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.644007 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.644351 (+ 344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.645869 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.645870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.645871 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.646072 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.646259 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.646259 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.646260 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.646317 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.646520 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.646521 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.646561 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.646635 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.646784 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6902,"mutex_wait_us":65,"thread_start_us":54,"threads_started":1}
I20260925 13:07:36.657786 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.634084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34212 (local address 127.0.206.126:38937)
0925 13:07:36.634122 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.634124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.634477 (+ 353us) server_negotiation.cc:407] Connection header received
0925 13:07:36.634548 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.634549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.634559 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.634607 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.634999 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.635718 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.636645 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.636858 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.636941 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.655149 (+ 18208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.655151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.655151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.655159 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.656576 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.656960 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.656961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.656961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.657020 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.657239 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.657240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.657241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.657362 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.657443 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.657613 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.657685 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 13:07:36.657796 7679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.634036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:34212)
0925 13:07:36.634381 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.634392 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.634489 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.634613 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.634614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.634825 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.634909 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.634911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.635809 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.635810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.636534 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.636536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.636949 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.636950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.637008 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.637791 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.637795 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.655028 (+ 17233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.656692 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.656693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.656693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.656870 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.657095 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.657095 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.657096 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.657152 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.657435 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.657436 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.657492 (+ 56us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.657585 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.657687 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":80,"threads_started":1}
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341655, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.126@KRBTEST.COM for oryx/127.0.206.124@KRBTEST.COM
I20260925 13:07:36.666764 7659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.660081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34694 (local address 127.0.206.124:36989)
0925 13:07:36.660140 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.660142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.660150 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:36.660180 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.660181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.660190 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.660257 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.660490 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.661206 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.661920 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.662236 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.662304 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.664305 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.664306 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.664306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.664313 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.665717 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.666038 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.666039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.666039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.666093 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.666271 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.666272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.666272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.666402 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.666474 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.666570 (+ 96us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.666638 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 13:07:36.666680 7494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.659996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:36989 (local address 127.0.0.1:34694)
0925 13:07:36.660050 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.660062 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.660102 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.660258 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.660259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.660380 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.660417 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.660419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.661288 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.661289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.661849 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.661850 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.662311 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.662312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.662360 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.662798 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.662803 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.664215 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.665800 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.665801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.665801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.665962 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.666159 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.666160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.666160 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.666208 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.666465 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.666466 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.666498 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.666548 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.666619 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 13:07:36.667428 6937 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.206.124:36989: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab"
member_type: NON_VOTER
last_known_addr {
host: "127.0.206.124"
port: 36989
}
attrs {
promote: true
}
}
dest_uuid: "0b09bc81bfcd4976a0b4dc056fa50948"
cas_config_opid_index: 5
I20260925 13:07:36.667711 6937 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: true } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: true } }
I20260925 13:07:36.668350 7630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:36.668380 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 6999
I20260925 13:07:36.669238 7630 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 1 FOLLOWER]: Refusing update from remote peer 0b09bc81bfcd4976a0b4dc056fa50948: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 13:07:36.669282 7495 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8895dce2e73c44d9be634ce437bfbaab (127.0.206.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: true } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: true } } }
I20260925 13:07:36.669432 7661 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:36.669510 7661 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } }
I20260925 13:07:36.669723 7661 leader_election.cc:288] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0b09bc81bfcd4976a0b4dc056fa50948 (127.0.206.126:38937), 0ca60ab3dcb2456395272077367c4e44 (127.0.206.125:43157)
I20260925 13:07:36.669912 6947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8895dce2e73c44d9be634ce437bfbaab" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" is_pre_election: true
I20260925 13:07:36.670229 7495 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.206.124:36989
I20260925 13:07:36.670305 7496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: true } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: true } } }
I20260925 13:07:36.670387 7496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:36.670473 7544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8895dce2e73c44d9be634ce437bfbaab" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ca60ab3dcb2456395272077367c4e44" is_pre_election: true
I20260925 13:07:36.670567 7588 master.cc:537] Master@127.0.206.124:36989 shutting down...
I20260925 13:07:36.670549 7496 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:36.670580 7544 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 LEARNER]: Handling vote request from an unknown peer 8895dce2e73c44d9be634ce437bfbaab
I20260925 13:07:36.670634 7544 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 8895dce2e73c44d9be634ce437bfbaab for term 2 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260925 13:07:36.670908 7607 leader_election.cc:302] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8895dce2e73c44d9be634ce437bfbaab; no voters: 0b09bc81bfcd4976a0b4dc056fa50948, 0ca60ab3dcb2456395272077367c4e44
I20260925 13:07:36.671092 7661 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 13:07:36.671402 6924 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 -> Peer 8895dce2e73c44d9be634ce437bfbaab (127.0.206.124:36989): Couldn't send request to peer 8895dce2e73c44d9be634ce437bfbaab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 13:07:36.674842 7494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.668222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43157 (local address 127.0.0.1:34770)
0925 13:07:36.671158 (+ 2936us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.671169 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.671224 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.671527 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.671528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.671529 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:36.671568 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.671570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.672665 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.672666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.673706 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.673708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.674540 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.674541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.674611 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.674635 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.674742 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.674783 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2921}
I20260925 13:07:36.674885 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.668343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34770 (local address 127.0.206.125:43157)
0925 13:07:36.668410 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.668412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.671369 (+ 2957us) server_negotiation.cc:407] Connection header received
0925 13:07:36.671393 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.671394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.671417 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.671473 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:36.671654 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.672474 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.673771 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.674448 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.674516 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.674591 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.674745 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.674791 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 13:07:36.675228 7544 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 LEARNER]: Refusing update from remote peer 0b09bc81bfcd4976a0b4dc056fa50948: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 13:07:36.675449 7496 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 13:07:36.675980 7544 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8895dce2e73c44d9be634ce437bfbaab (127.0.206.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: true } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: true } } }
I20260925 13:07:36.677014 7682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: true } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: true } } }
I20260925 13:07:36.677111 7682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [sys.catalog]: This master's current role is: LEARNER
I20260925 13:07:36.677217 7574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b09bc81bfcd4976a0b4dc056fa50948. Latest consensus state: current_term: 1 leader_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: true } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: true } } }
I20260925 13:07:36.677315 7574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [sys.catalog]: This master's current role is: LEARNER
I20260925 13:07:36.677407 7588 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:36.677484 7588 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:36.677521 7588 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: stopping tablet replica
I20260925 13:07:36.677583 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:45033
--local_ip_for_outbound_sockets=127.0.206.65
--tserver_master_addrs=127.0.206.126:38937
--webserver_port=43307
--webserver_interface=127.0.206.65
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.65.keytab
--principal=oryx/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.206.126:38937,127.0.206.125:43157
--tserver_master_addrs=127.0.206.126:38937,127.0.206.125:43157,127.0.206.124:36989 with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:07:36.679116 7496 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948: attempting to promote NON_VOTER 0ca60ab3dcb2456395272077367c4e44 to VOTER
I20260925 13:07:36.679356 7496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: true } }
I20260925 13:07:36.680249 7544 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 LEARNER]: Refusing update from remote peer 0b09bc81bfcd4976a0b4dc056fa50948: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260925 13:07:36.680446 7495 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 13:07:36.681062 7495 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0ca60ab3dcb2456395272077367c4e44 (127.0.206.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: true } } }
I20260925 13:07:36.682097 7495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: true } } }
I20260925 13:07:36.682191 7495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:36.682250 7495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: true } } }
I20260925 13:07:36.682322 7495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:36.682602 7544 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0ca60ab3dcb2456395272077367c4e44 (127.0.206.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: true } } }
I20260925 13:07:36.682870 7494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.679901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:36989 (local address 127.0.0.1:34700)
0925 13:07:36.682791 (+ 2890us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.682840 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2872}
I20260925 13:07:36.683670 7574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: true } } }
I20260925 13:07:36.683744 7574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:07:36.684049 7494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.683935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:36989 (local address 127.0.0.1:34716)
0925 13:07:36.683994 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.684026 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:36989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260925 13:07:36.684196 6924 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 -> Peer 8895dce2e73c44d9be634ce437bfbaab (127.0.206.124:36989): Couldn't send request to peer 8895dce2e73c44d9be634ce437bfbaab. Status: Network error: Client connection negotiation failed: client connection to 127.0.206.124:36989: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 13:07:36.690846 7588 master.cc:552] Master@127.0.206.124:36989 shutdown complete.
I20260925 13:07:36.690901 7588 master_runner.cc:305] Clearing existing system tablet
I20260925 13:07:36.691032 7588 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:36.691079 7588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:36.691792 7588 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:07:36.692368 7686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:36.692492 7588 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:36.692550 7588 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:07:36.692574 7588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/wal
uuid: "8895dce2e73c44d9be634ce437bfbaab"
format_stamp: "Formatted at 2026-09-25 13:07:36 on dist-test-slave-chkf"
server_key: "4ea41f4e8b7605cb7f7085bda3d07944"
server_key_iv: "9b26ed86ce06375f1121b994662c1f14"
server_key_version: "encryptionkey@0"
I20260925 13:07:36.693329 7588 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:36.694927 7588 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 13:07:36.694998 7588 log.cc:1196] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260925 13:07:36.695192 7588 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: Deleting consensus metadata
I20260925 13:07:36.695318 7588 master_runner.cc:315] Copying system tablet from 127.0.206.126:38937
I20260925 13:07:36.695905 7588 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: tablet copy: Beginning tablet copy session from remote peer at address 127.0.206.126:38937
I20260925 13:07:36.705911 7692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.696208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:34226)
0925 13:07:36.698811 (+ 2603us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.698824 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.698901 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.699038 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.699039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.699242 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.699316 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.699318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.700255 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.700256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.701106 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.701109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.701632 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.701633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.701726 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.702233 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.702239 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.702634 (+ 395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.704704 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.704705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.704706 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.704939 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.705185 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.705185 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.705186 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.705256 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.705560 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.705561 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.705613 (+ 52us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.705688 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.705797 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2582,"thread_start_us":88,"threads_started":1}
I20260925 13:07:36.705940 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.696273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34226 (local address 127.0.206.126:38937)
0925 13:07:36.696315 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.696317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.698915 (+ 2598us) server_negotiation.cc:407] Connection header received
0925 13:07:36.698953 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.698954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.698966 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.699036 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.699398 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.700155 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.701211 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.701506 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.701576 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.702804 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.702806 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.702806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.702814 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.704493 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.705027 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.705028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.705028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.705096 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.705335 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.705336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.705337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.705476 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.705546 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.705734 (+ 188us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.705812 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 13:07:36.706602 6957 tablet_copy_service.cc:140] P 0b09bc81bfcd4976a0b4dc056fa50948: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8895dce2e73c44d9be634ce437bfbaab ({username='oryx', principal='oryx/127.0.206.124@KRBTEST.COM'} at 127.0.0.1:34226)
I20260925 13:07:36.706660 6957 tablet_copy_service.cc:161] P 0b09bc81bfcd4976a0b4dc056fa50948: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8895dce2e73c44d9be634ce437bfbaab at {username='oryx', principal='oryx/127.0.206.124@KRBTEST.COM'} at 127.0.0.1:34226: session id = 8895dce2e73c44d9be634ce437bfbaab-00000000000000000000000000000000
I20260925 13:07:36.707269 6957 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 13:07:36.707890 7588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 13:07:36.709208 7588 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: tablet copy: Starting download of 0 data blocks...
I20260925 13:07:36.709348 7588 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: tablet copy: Starting download of 1 WAL segments...
I20260925 13:07:36.710091 7588 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 13:07:36.710989 6957 tablet_copy_service.cc:342] P 0b09bc81bfcd4976a0b4dc056fa50948: Request end of tablet copy session 8895dce2e73c44d9be634ce437bfbaab-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.206.124@KRBTEST.COM'} at 127.0.0.1:34226
I20260925 13:07:36.711043 6957 tablet_copy_service.cc:433] P 0b09bc81bfcd4976a0b4dc056fa50948: ending tablet copy session 8895dce2e73c44d9be634ce437bfbaab-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8895dce2e73c44d9be634ce437bfbaab
I20260925 13:07:36.713408 7588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:36.714754 7693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:36.719689 7694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:36.719868 7696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:36.720031 7588 server_base.cc:1034] running on GCE node
I20260925 13:07:36.720115 7588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:36.720255 7588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:36.721397 7588 hybrid_clock.cc:648] HybridClock initialized: now 1790341656721387 us; error 29 us; skew 500 ppm
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:36.723035 7588 init.cc:377] Logged in from keytab as oryx/127.0.206.124@KRBTEST.COM (short username oryx)
I20260925 13:07:36.723304 7588 webserver.cc:460] Webserver started at http://127.0.206.124:44413/ using document root <none> and password file <none>
I20260925 13:07:36.723433 7588 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:36.723471 7588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:36.724249 7588 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:36.724892 7703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:36.725106 7588 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 13:07:36.725159 7588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/wal
uuid: "8895dce2e73c44d9be634ce437bfbaab"
format_stamp: "Formatted at 2026-09-25 13:07:36 on dist-test-slave-chkf"
server_key: "4ea41f4e8b7605cb7f7085bda3d07944"
server_key_iv: "9b26ed86ce06375f1121b994662c1f14"
server_key_version: "encryptionkey@0"
I20260925 13:07:36.725220 7588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:36.735873 7588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:36.737753 7588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:36.740801 7588 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.124:36989
I20260925 13:07:36.741143 7588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/master-2/data/info.pb
I20260925 13:07:36.741994 7756 sys_catalog.cc:263] Verifying existing consensus state
I20260925 13:07:36.742504 7756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: Bootstrap starting.
I20260925 13:07:36.743798 7755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.124:36989 every 8 connection(s)
I20260925 13:07:36.745498 7756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:07:36.745734 7756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: Bootstrap complete.
I20260925 13:07:36.745970 7756 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: true } }
I20260925 13:07:36.746068 7756 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8895dce2e73c44d9be634ce437bfbaab, State: Initialized, Role: LEARNER
I20260925 13:07:36.746109 7756 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: true } }
I20260925 13:07:36.746284 7756 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:36.747835 7770 catalog_manager.cc:1259] Loaded cluster ID: d4ab679f95aa42aaadb7f7d3c98d2d00
I20260925 13:07:36.747924 7770 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: loading cluster ID for follower catalog manager: success
I20260925 13:07:36.748157 7759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: true } } }
I20260925 13:07:36.748246 7759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [sys.catalog]: This master's current role is: LEARNER
I20260925 13:07:36.749804 7770 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: acquiring CA information for follower catalog manager: success
I20260925 13:07:36.750097 7770 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 13:07:36.775555 7683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:36.775753 7683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:36.775784 7683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:36.776966 7683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:36.777009 7683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:36.777034 7683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:07:36.778527 7683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.65.keytab
--principal=oryx/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:45033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=43307
--tserver_master_addrs=127.0.206.126:38937,127.0.206.125:43157,127.0.206.124:36989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:36.778795 7683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:36.779068 7683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:36.785529 7776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:36.785663 7778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:36.785557 7683 server_base.cc:1034] running on GCE node
W20260925 13:07:36.785513 7775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:36.785929 7683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:36.786181 7683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:36.787339 7683 hybrid_clock.cc:648] HybridClock initialized: now 1790341656787328 us; error 29 us; skew 500 ppm
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:36.790288 7683 init.cc:377] Logged in from keytab as oryx/127.0.206.65@KRBTEST.COM (short username oryx)
I20260925 13:07:36.790778 7683 webserver.cc:460] Webserver started at http://127.0.206.65:43307/ using document root <none> and password file <none>
I20260925 13:07:36.791028 7683 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:36.791117 7683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:36.792514 7683 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:36.793187 7785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:36.793407 7683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:36.793525 7683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a"
format_stamp: "Formatted at 2026-09-25 13:07:35 on dist-test-slave-chkf"
server_key: "3123f7d770dff503d443bccaabe149b9"
server_key_iv: "3b37c43d7364cb755e79b8795429330c"
server_key_version: "encryptionkey@0"
I20260925 13:07:36.793887 7683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:36.807588 7683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:36.809399 7683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:36.809568 7683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:36.809859 7683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:36.810186 7683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:36.810248 7683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:36.810295 7683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:36.810324 7683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:36.817185 7683 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:45033
I20260925 13:07:36.817219 7898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:45033 every 8 connection(s)
I20260925 13:07:36.817585 7683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:07:36.823972 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 7683
I20260925 13:07:36.824137 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 7133
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.125@KRBTEST.COM
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.124@KRBTEST.COM
I20260925 13:07:36.829986 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:32855
--local_ip_for_outbound_sockets=127.0.206.66
--tserver_master_addrs=127.0.206.126:38937
--webserver_port=39195
--webserver_interface=127.0.206.66
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.66.keytab
--principal=oryx/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.206.126:38937,127.0.206.125:43157
--tserver_master_addrs=127.0.206.126:38937,127.0.206.125:43157,127.0.206.124:36989 with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260925 13:07:36.830246 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.818803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:44729 (local address 127.0.206.125:43157)
0925 13:07:36.818852 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.818854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.818971 (+ 117us) server_negotiation.cc:407] Connection header received
0925 13:07:36.819534 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.819535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.819546 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.819600 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.820565 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.821063 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.822462 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.822683 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.822815 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.825905 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.825906 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.825907 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.825914 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.828298 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.828888 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.828912 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.828912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.828968 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.829348 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.829349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.829350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.829507 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.829556 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.830018 (+ 462us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.830113 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 13:07:36.830488 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.818690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43157 (local address 127.0.206.65:44729)
0925 13:07:36.818889 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.818899 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.819381 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.819692 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.819696 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.820254 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.820471 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.820475 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.821172 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.821173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.822361 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.822362 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.822824 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.822825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.822948 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.823546 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.823556 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.825790 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.828395 (+ 2605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.828402 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.828405 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.828777 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.829077 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.829078 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.829078 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.829218 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.829677 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.829683 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.829848 (+ 165us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.829971 (+ 123us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.830192 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":27,"thread_start_us":150,"threads_started":1}
I20260925 13:07:36.830976 7901 heartbeater.cc:347] Connected to a master server at 127.0.206.125:43157
I20260925 13:07:36.831081 7901 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:36.831290 7901 heartbeater.cc:511] Master 127.0.206.125:43157 requested a full tablet report, sending...
I20260925 13:07:36.831813 7534 ts_manager.cc:194] Registered new tserver with Master: 5fc6c6780d8f4d14b8fe491aa3bc687a (127.0.206.65:45033)
I20260925 13:07:36.833596 7905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.818971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:36989 (local address 127.0.206.65:36175)
0925 13:07:36.819103 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.819113 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.819387 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.819692 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.819696 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.820249 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.820472 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.820477 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.821767 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.821768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.822356 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.822367 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.823125 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.823126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.823186 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.823842 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.823848 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.828638 (+ 4790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.832202 (+ 3564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.832203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.832203 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.832440 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.832639 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.832640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.832640 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.832692 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.833254 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.833255 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.833299 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.833366 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.833473 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":8,"thread_start_us":62,"threads_started":1}
I20260925 13:07:36.834033 7906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.819026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:36175 (local address 127.0.206.124:36989)
0925 13:07:36.819146 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.819148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.819264 (+ 116us) server_negotiation.cc:407] Connection header received
0925 13:07:36.819542 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.819543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.819557 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.819613 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.820579 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.821681 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.822799 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.823043 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.823094 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.828887 (+ 5793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.828888 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.828889 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.828897 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.832122 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.832518 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.832519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.832520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.832582 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.833044 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.833045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.833045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.833150 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.833252 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.833818 (+ 566us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.833910 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":53,"threads_started":1}
I20260925 13:07:36.834407 7900 heartbeater.cc:347] Connected to a master server at 127.0.206.124:36989
I20260925 13:07:36.834455 7900 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:36.834527 7900 heartbeater.cc:511] Master 127.0.206.124:36989 requested a full tablet report, sending...
I20260925 13:07:36.834869 7720 ts_manager.cc:194] Registered new tserver with Master: 5fc6c6780d8f4d14b8fe491aa3bc687a (127.0.206.65:45033)
I20260925 13:07:36.835269 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.818419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:40911 (local address 127.0.206.126:38937)
0925 13:07:36.818519 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.818522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.818753 (+ 231us) server_negotiation.cc:407] Connection header received
0925 13:07:36.819495 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.819496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.819511 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.819567 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.820579 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.821463 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.822486 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.822682 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.822813 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.832255 (+ 9442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.832257 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.832258 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.832266 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.833972 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.834412 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.834414 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.834414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.834478 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.834682 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.834683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.834684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.834840 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.834914 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.835135 (+ 221us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.835211 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 13:07:36.835395 7903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.818379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.65:40911)
0925 13:07:36.818553 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.818589 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.819355 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.819692 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.819697 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.820248 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.820472 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.820477 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.821561 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.821562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.822360 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.822362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.823156 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.823156 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.823211 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.828758 (+ 5547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.828763 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.831375 (+ 2612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.834062 (+ 2687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.834063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.834064 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.834294 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.834559 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.834559 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.834560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.834623 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.834924 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.834925 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.834985 (+ 60us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.835100 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.835212 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":3,"thread_start_us":101,"threads_started":1}
I20260925 13:07:36.835767 7899 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38937
I20260925 13:07:36.835816 7899 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:36.835877 7899 heartbeater.cc:511] Master 127.0.206.126:38937 requested a full tablet report, sending...
I20260925 13:07:36.836185 6937 ts_manager.cc:194] Re-registered known tserver with Master: 5fc6c6780d8f4d14b8fe491aa3bc687a (127.0.206.65:45033)
I20260925 13:07:36.836563 6937 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:40911
W20260925 13:07:36.912176 7907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:36.912366 7907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:36.912389 7907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:36.913609 7907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:36.913648 7907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:36.913704 7907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:07:36.915169 7907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.66.keytab
--principal=oryx/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:32855
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=39195
--tserver_master_addrs=127.0.206.126:38937,127.0.206.125:43157,127.0.206.124:36989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:36.915428 7907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:36.915663 7907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:36.920677 7915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:36.920713 7913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:36.920832 7907 server_base.cc:1034] running on GCE node
W20260925 13:07:36.920893 7912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:36.921128 7907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:36.921386 7907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:36.922537 7907 hybrid_clock.cc:648] HybridClock initialized: now 1790341656922526 us; error 29 us; skew 500 ppm
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:36.925227 7907 init.cc:377] Logged in from keytab as oryx/127.0.206.66@KRBTEST.COM (short username oryx)
I20260925 13:07:36.925645 7907 webserver.cc:460] Webserver started at http://127.0.206.66:39195/ using document root <none> and password file <none>
I20260925 13:07:36.925900 7907 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:36.925956 7907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:36.927397 7907 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260925 13:07:36.928018 7922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:36.928279 7907 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:36.928352 7907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d"
format_stamp: "Formatted at 2026-09-25 13:07:35 on dist-test-slave-chkf"
server_key: "aa89670db4d688e4aca1d9e95dde4507"
server_key_iv: "2e115c3e2cb3dca81a669a3c0d00f1ea"
server_key_version: "encryptionkey@0"
I20260925 13:07:36.928651 7907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:36.939052 7907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:36.940744 7907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:36.940853 7907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:36.941007 7907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:36.941229 7907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:36.941257 7907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:36.941277 7907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:36.941289 7907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:36.947115 7907 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:32855
I20260925 13:07:36.947191 8035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:32855 every 8 connection(s)
I20260925 13:07:36.947511 7907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.124@KRBTEST.COM
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.125@KRBTEST.COM
I20260925 13:07:36.955421 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 7907
Sep 25 13:07:36 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341656, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260925 13:07:36.955607 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 7267
I20260925 13:07:36.958866 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.948765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:35085 (local address 127.0.206.125:43157)
0925 13:07:36.948816 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.948818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.948941 (+ 123us) server_negotiation.cc:407] Connection header received
0925 13:07:36.949362 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.949363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.949377 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.949430 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.950059 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.950859 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.951989 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.952286 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.952354 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.955454 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.955455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.955456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.955463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.957360 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.957847 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.957848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.957849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.957915 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.958190 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.958191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.958191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.958302 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.958397 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.958675 (+ 278us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.958772 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 13:07:36.959154 8042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.948718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43157 (local address 127.0.206.66:35085)
0925 13:07:36.948860 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.948869 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.949215 (+ 346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.949453 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.949454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.949777 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.949957 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.949961 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.950943 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.950944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.951847 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.951849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.952371 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.952371 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.952597 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.953183 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.953193 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.955304 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.957463 (+ 2159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.957469 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.957471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.957760 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.958008 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.958009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.958009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.958132 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.958391 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.958396 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.958534 (+ 138us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.958637 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.958837 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":65,"threads_started":1}
I20260925 13:07:36.959158 8041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.948552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.66:46963)
0925 13:07:36.948675 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.948685 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.949200 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.949558 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.949559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.949822 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.949958 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.949964 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.950766 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.950766 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.952010 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.952011 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.952458 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.952459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.952597 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.953494 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.953499 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.955875 (+ 2376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.957632 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.957634 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.957634 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.957891 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.958117 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.958118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.958118 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.958192 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.958509 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.958510 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.958552 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.958628 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.958947 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":132,"thread_start_us":63,"threads_started":1}
I20260925 13:07:36.959383 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.948617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:46963 (local address 127.0.206.126:38937)
0925 13:07:36.948664 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.948666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.948738 (+ 72us) server_negotiation.cc:407] Connection header received
0925 13:07:36.949461 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.949462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.949473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.949532 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.950068 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.950608 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.952093 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.952382 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.952443 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.955961 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.955963 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.955963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.955970 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.957546 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.957979 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.957981 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.957981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.958037 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.958277 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.958278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.958279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.958418 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.958462 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.959182 (+ 720us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.959262 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 13:07:36.959609 8038 heartbeater.cc:347] Connected to a master server at 127.0.206.125:43157
I20260925 13:07:36.959692 8038 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:36.959867 8038 heartbeater.cc:511] Master 127.0.206.125:43157 requested a full tablet report, sending...
I20260925 13:07:36.960301 7534 ts_manager.cc:194] Registered new tserver with Master: a1ea26f9b21b4a16af8b2434fd43cc7d (127.0.206.66:32855)
I20260925 13:07:36.961350 8036 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38937
I20260925 13:07:36.961406 8036 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:36.961488 8036 heartbeater.cc:511] Master 127.0.206.126:38937 requested a full tablet report, sending...
I20260925 13:07:36.962764 6937 ts_manager.cc:194] Re-registered known tserver with Master: a1ea26f9b21b4a16af8b2434fd43cc7d (127.0.206.66:32855)
I20260925 13:07:36.963083 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:37123
--local_ip_for_outbound_sockets=127.0.206.67
--tserver_master_addrs=127.0.206.126:38937
--webserver_port=35567
--webserver_interface=127.0.206.67
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.67.keytab
--principal=oryx/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.206.126:38937,127.0.206.125:43157
--tserver_master_addrs=127.0.206.126:38937,127.0.206.125:43157,127.0.206.124:36989 with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:07:36.963243 6937 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:46963
I20260925 13:07:36.965035 7906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.948297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:36625 (local address 127.0.206.124:36989)
0925 13:07:36.948366 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:36.948368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:36.948570 (+ 202us) server_negotiation.cc:407] Connection header received
0925 13:07:36.949308 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:36.949309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:36.949324 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:36.949381 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:36.950189 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.950976 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.952496 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.952689 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.952789 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.954889 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:36.954890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:36.954891 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:36.954897 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:36.963801 (+ 8904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.964172 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.964173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.964174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.964240 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:36.964488 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:36.964489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:36.964490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:36.964599 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:36.964657 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:36.964880 (+ 223us) server_negotiation.cc:299] Negotiation successful
0925 13:07:36.964975 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 13:07:36.965083 8040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:36.948281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:36989 (local address 127.0.206.66:36625)
0925 13:07:36.948453 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:36.948486 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:36.949199 (+ 713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:36.949447 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:36.949451 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:36.950055 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:36.950106 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.950107 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.951055 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.951055 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.952004 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:36.952007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:36.952794 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:36.952795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:36.952859 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:36.953356 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:36.953366 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:36.954802 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:36.963911 (+ 9109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.963912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.963912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.964080 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.964346 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:36.964347 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:36.964348 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:36.964425 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:36.964675 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:36.964676 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:36.964723 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 13:07:36.964809 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 13:07:36.964940 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260925 13:07:36.965444 8037 heartbeater.cc:347] Connected to a master server at 127.0.206.124:36989
I20260925 13:07:36.965489 8037 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:36.965548 8037 heartbeater.cc:511] Master 127.0.206.124:36989 requested a full tablet report, sending...
I20260925 13:07:36.965850 7720 ts_manager.cc:194] Registered new tserver with Master: a1ea26f9b21b4a16af8b2434fd43cc7d (127.0.206.66:32855)
W20260925 13:07:37.044946 8043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:37.045135 8043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:37.045156 8043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:37.046290 8043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:37.046331 8043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:37.046357 8043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:07:37.048018 8043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/krb5kdc/oryx_127.0.206.67.keytab
--principal=oryx/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:37123
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=35567
--tserver_master_addrs=127.0.206.126:38937,127.0.206.125:43157,127.0.206.124:36989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:37.048252 8043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:37.048444 8043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:37.053403 8051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:37.053378 8048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:37.053367 8049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:37.053637 8043 server_base.cc:1034] running on GCE node
I20260925 13:07:37.053891 8043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:37.054186 8043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:37.055341 8043 hybrid_clock.cc:648] HybridClock initialized: now 1790341657055328 us; error 27 us; skew 500 ppm
Sep 25 13:07:37 dist-test-slave-chkf krb5kdc[6892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341657, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:37.057586 8043 init.cc:377] Logged in from keytab as oryx/127.0.206.67@KRBTEST.COM (short username oryx)
I20260925 13:07:37.058039 8043 webserver.cc:460] Webserver started at http://127.0.206.67:35567/ using document root <none> and password file <none>
I20260925 13:07:37.058272 8043 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:37.058357 8043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:37.059736 8043 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:37.060312 8058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:37.060492 8043 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:37.060590 8043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "79e6f0ab9db24450a82431ce46059fda"
format_stamp: "Formatted at 2026-09-25 13:07:36 on dist-test-slave-chkf"
server_key: "762953716b1a67d053aef2e51533bb44"
server_key_iv: "a5af32e401fa284bb21d02a162eadc96"
server_key_version: "encryptionkey@0"
I20260925 13:07:37.060935 8043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:37.077685 8043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:37.079926 8043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:37.080101 8043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:37.080331 8043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:37.080646 8043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:37.080704 8043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:37.080739 8043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:37.080765 8043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:37.086537 8043 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:37123
I20260925 13:07:37.086581 8171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:37123 every 8 connection(s)
I20260925 13:07:37.086970 8043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:07:37.088441 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 8043
Sep 25 13:07:37 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341657, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
Sep 25 13:07:37 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341657, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.125@KRBTEST.COM
Sep 25 13:07:37 dist-test-slave-chkf krb5kdc[6892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341657, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.124@KRBTEST.COM
I20260925 13:07:37.100347 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.088346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:36365 (local address 127.0.206.125:43157)
0925 13:07:37.088384 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:37.088386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:37.088395 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:37.088840 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:37.088841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:37.088853 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:37.088906 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:37.089780 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.090470 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.091953 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.092256 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.092293 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.096780 (+ 4487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:37.096782 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:37.096782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:37.096789 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:37.098506 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:37.099206 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:37.099207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:37.099208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:37.099265 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:37.099571 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:37.099573 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:37.099573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:37.099706 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:37.099759 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:37.100153 (+ 394us) server_negotiation.cc:299] Negotiation successful
0925 13:07:37.100228 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 13:07:37.101123 8178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.088174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43157 (local address 127.0.206.67:36365)
0925 13:07:37.088312 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:37.088322 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:37.088581 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:37.088944 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:37.088945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:37.089363 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:37.089665 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:37.089671 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.090610 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.090611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:37.091849 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:37.091851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.092378 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.092379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:37.092517 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.093376 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:37.093382 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:37.096650 (+ 3268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:37.098656 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:37.098663 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:37.098665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:37.099112 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:37.099345 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:37.099345 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:37.099346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:37.099484 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:37.099794 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:37.099800 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:37.099981 (+ 181us) client_negotiation.cc:769] Sending connection context
0925 13:07:37.100679 (+ 698us) client_negotiation.cc:240] Negotiation successful
0925 13:07:37.100854 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":25,"thread_start_us":74,"threads_started":1}
I20260925 13:07:37.101876 8174 heartbeater.cc:347] Connected to a master server at 127.0.206.125:43157
I20260925 13:07:37.101980 8174 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:37.102237 8174 heartbeater.cc:511] Master 127.0.206.125:43157 requested a full tablet report, sending...
I20260925 13:07:37.102296 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.087687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:41105 (local address 127.0.206.126:38937)
0925 13:07:37.087779 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:37.087781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:37.088010 (+ 229us) server_negotiation.cc:407] Connection header received
0925 13:07:37.088673 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:37.088675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:37.088689 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:37.088781 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:37.089778 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.091266 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.092228 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.092516 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.092554 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.096326 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:37.096328 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:37.096328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:37.096335 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:37.099152 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:37.101333 (+ 2181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:37.101334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:37.101334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:37.101384 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:37.101600 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:37.101602 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:37.101602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:37.101743 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:37.101806 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:37.102118 (+ 312us) server_negotiation.cc:299] Negotiation successful
0925 13:07:37.102203 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 13:07:37.102418 7906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.088140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:50517 (local address 127.0.206.124:36989)
0925 13:07:37.088214 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:37.088217 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:37.088226 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:37.089045 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:37.089046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:37.089058 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:37.090105 (+ 1047us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:37.093258 (+ 3153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.094173 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.095347 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.096014 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.096061 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.099288 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:37.099289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:37.099289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:37.099296 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:37.101046 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:37.101435 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:37.101436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:37.101436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:37.101496 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:37.101707 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:37.101708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:37.101708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:37.101838 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:37.101883 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:37.102244 (+ 361us) server_negotiation.cc:299] Negotiation successful
0925 13:07:37.102328 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 13:07:37.102648 8176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.087680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.67:41105)
0925 13:07:37.087865 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:37.087902 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:37.088577 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:37.088842 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:37.088845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:37.089444 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:37.089665 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:37.089671 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.091360 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.091361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:37.092146 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:37.092148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.092627 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.092628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:37.092695 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.093305 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:37.093316 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:37.096246 (+ 2930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:37.100438 (+ 4192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:37.100439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:37.100440 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:37.100648 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:37.101467 (+ 819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:37.101468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:37.101468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:37.101524 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:37.101937 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:37.101938 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:37.101986 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 13:07:37.102431 (+ 445us) client_negotiation.cc:240] Negotiation successful
0925 13:07:37.102530 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":3,"thread_start_us":119,"threads_started":1}
I20260925 13:07:37.102675 7534 ts_manager.cc:194] Registered new tserver with Master: 79e6f0ab9db24450a82431ce46059fda (127.0.206.67:37123)
I20260925 13:07:37.102927 8177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.088016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:36989 (local address 127.0.206.67:50517)
0925 13:07:37.088134 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:37.088144 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:37.088957 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:37.090190 (+ 1233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:37.090190 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:37.090391 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:37.090463 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:37.090465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.094272 (+ 3807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.094273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:37.095243 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:37.095246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.096137 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.096137 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:37.096210 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.097643 (+ 1433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:37.097649 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:37.099191 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:37.101124 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:37.101125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:37.101125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:37.101355 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:37.101573 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:37.101574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:37.101575 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:37.101634 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:37.101931 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:37.101932 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:37.102070 (+ 138us) client_negotiation.cc:769] Sending connection context
0925 13:07:37.102541 (+ 471us) client_negotiation.cc:240] Negotiation successful
0925 13:07:37.102810 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":60,"threads_started":1}
I20260925 13:07:37.103366 8172 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38937
I20260925 13:07:37.103425 8172 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:37.103482 8173 heartbeater.cc:347] Connected to a master server at 127.0.206.124:36989
I20260925 13:07:37.103500 8172 heartbeater.cc:511] Master 127.0.206.126:38937 requested a full tablet report, sending...
I20260925 13:07:37.103511 8173 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:37.103569 8173 heartbeater.cc:511] Master 127.0.206.124:36989 requested a full tablet report, sending...
I20260925 13:07:37.103793 6937 ts_manager.cc:194] Re-registered known tserver with Master: 79e6f0ab9db24450a82431ce46059fda (127.0.206.67:37123)
I20260925 13:07:37.103874 7720 ts_manager.cc:194] Registered new tserver with Master: 79e6f0ab9db24450a82431ce46059fda (127.0.206.67:37123)
I20260925 13:07:37.104422 6937 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:41105
I20260925 13:07:37.105144 8183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.089593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34230 (local address 127.0.206.126:38937)
0925 13:07:37.090904 (+ 1311us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:37.090906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:37.090913 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:07:37.090931 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:37.090932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:37.090942 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:37.090999 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:37.091971 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.092821 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.095045 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.095348 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.095408 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.097271 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:37.097272 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:37.097272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:37.097582 (+ 310us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:37.101224 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:37.103369 (+ 2145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:37.103370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:37.103371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:37.103448 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:37.103672 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:37.103673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:37.103673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:37.103791 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:37.103893 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:37.104929 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0925 13:07:37.105018 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1286,"thread_start_us":86,"threads_started":1}
I20260925 13:07:37.106208 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.090091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.0.206.125:43157)
0925 13:07:37.092850 (+ 2759us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:37.092852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:37.092860 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:37.092880 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:37.092881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:37.092894 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:37.092958 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:37.094174 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.094942 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.095945 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.096246 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.096312 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.097793 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:37.097794 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:37.097794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:37.097801 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:37.104451 (+ 6650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:37.104873 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:37.104874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:37.104875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:37.104941 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:37.105133 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:37.105134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:37.105135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:37.105505 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:37.105603 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:37.105695 (+ 92us) server_negotiation.cc:299] Negotiation successful
0925 13:07:37.105773 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2740,"thread_start_us":51,"threads_started":1}
I20260925 13:07:37.108165 8187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.090409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34730 (local address 127.0.206.124:36989)
0925 13:07:37.090687 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:37.090689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:37.098317 (+ 7628us) server_negotiation.cc:407] Connection header received
0925 13:07:37.098420 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:37.098421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:37.098433 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:37.098474 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:37.099189 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.099920 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.101114 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.101424 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.102849 (+ 1425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.105030 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:37.105031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:37.105032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:37.105038 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:37.106868 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:37.107218 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:37.107220 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:37.107220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:37.107272 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:37.107504 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:37.107505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:37.107506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:37.107618 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:37.107747 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:37.107987 (+ 240us) server_negotiation.cc:299] Negotiation successful
0925 13:07:37.108063 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":48,"threads_started":1}
I20260925 13:07:37.106824 6937 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:07:37.108415 6937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:07:37.115963 7494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.111983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:37123 (local address 127.0.0.1:44278)
0925 13:07:37.112014 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:37.112025 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:37.112061 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:37.112692 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:37.112693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:37.112694 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:37.112740 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:37.112742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.113974 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.113974 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:37.115002 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:37.115003 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.115705 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.115706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:37.115763 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.115768 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 13:07:37.115844 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 13:07:37.115887 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15,"mutex_wait_us":4}
I20260925 13:07:37.115984 8194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.112076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44278 (local address 127.0.206.67:37123)
0925 13:07:37.112541 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:37.112545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:37.112555 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:07:37.112590 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:37.112592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:37.112607 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:37.112680 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:37.112827 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.113885 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.115085 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.115618 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.115692 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.115763 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:37.115869 (+ 106us) server_negotiation.cc:299] Negotiation successful
0925 13:07:37.115914 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":82,"threads_started":1}
I20260925 13:07:37.116194 8193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.112076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:32855 (local address 127.0.0.1:50244)
0925 13:07:37.112192 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:37.112202 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:37.112418 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:37.112685 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:37.112686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:37.112689 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:37.112741 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:37.112742 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.113705 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.113706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:37.114974 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:37.114977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.115976 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.115977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:37.116038 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.116042 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:07:37.116096 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 13:07:37.116131 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":4,"thread_start_us":78,"threads_started":1}
I20260925 13:07:37.116818 8195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.112113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:45033 (local address 127.0.0.1:54352)
0925 13:07:37.112746 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:37.112754 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:37.112823 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:37.113092 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:37.113093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:37.113095 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:37.113137 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:37.113138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.114480 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.114481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:37.115458 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:37.115460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.116581 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.116582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:37.116643 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.116648 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 13:07:37.116697 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 13:07:37.116735 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":72,"threads_started":1}
I20260925 13:07:37.116822 8106 tablet_service.cc:1467] Processing CreateTablet for tablet f2ebed12913441149a6b003ef8108a47 (DEFAULT_TABLE table=test-table [id=c913ef2f53c5471f8668e3f6552bd337]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:37.117040 8197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.112343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54352 (local address 127.0.206.65:45033)
0925 13:07:37.112933 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:37.112937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:37.112946 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:37.112981 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:37.112983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:37.112997 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:37.113072 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:37.113197 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.114393 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.115557 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.116408 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.116479 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.116548 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:37.116921 (+ 373us) server_negotiation.cc:299] Negotiation successful
0925 13:07:37.116964 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":73,"threads_started":1}
I20260925 13:07:37.117050 8196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.112269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50244 (local address 127.0.206.66:32855)
0925 13:07:37.112434 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:37.112439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:37.112450 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:07:37.112491 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:37.112493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:37.112509 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:37.112590 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:37.112823 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.113623 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.115062 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.115814 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.115876 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.115943 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:37.116350 (+ 407us) server_negotiation.cc:299] Negotiation successful
0925 13:07:37.116399 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260925 13:07:37.117187 8106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2ebed12913441149a6b003ef8108a47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:37.117698 7833 tablet_service.cc:1467] Processing CreateTablet for tablet f2ebed12913441149a6b003ef8108a47 (DEFAULT_TABLE table=test-table [id=c913ef2f53c5471f8668e3f6552bd337]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:37.117686 7970 tablet_service.cc:1467] Processing CreateTablet for tablet f2ebed12913441149a6b003ef8108a47 (DEFAULT_TABLE table=test-table [id=c913ef2f53c5471f8668e3f6552bd337]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:37.118012 7833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2ebed12913441149a6b003ef8108a47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:37.118017 7970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2ebed12913441149a6b003ef8108a47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:37.120723 8198 tablet_bootstrap.cc:492] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a: Bootstrap starting.
I20260925 13:07:37.121050 8199 tablet_bootstrap.cc:492] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda: Bootstrap starting.
I20260925 13:07:37.121393 8198 tablet_bootstrap.cc:654] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:37.121639 8200 tablet_bootstrap.cc:492] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d: Bootstrap starting.
I20260925 13:07:37.121732 8198 log.cc:824] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:37.121832 8199 tablet_bootstrap.cc:654] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:37.122215 8200 tablet_bootstrap.cc:654] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:37.122578 8200 log.cc:824] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:37.122570 8199 log.cc:824] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:37.123198 8198 tablet_bootstrap.cc:492] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a: No bootstrap required, opened a new log
I20260925 13:07:37.123286 8198 ts_tablet_manager.cc:1397] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 13:07:37.123314 8199 tablet_bootstrap.cc:492] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda: No bootstrap required, opened a new log
I20260925 13:07:37.123371 8199 ts_tablet_manager.cc:1397] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 13:07:37.124872 8198 raft_consensus.cc:348] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } } peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } }
I20260925 13:07:37.125037 8198 raft_consensus.cc:374] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:37.125082 8198 raft_consensus.cc:729] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc6c6780d8f4d14b8fe491aa3bc687a, State: Initialized, Role: FOLLOWER
I20260925 13:07:37.125219 8198 consensus_queue.cc:260] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } } peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } }
I20260925 13:07:37.125169 8199 raft_consensus.cc:348] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } } peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } }
I20260925 13:07:37.125308 8199 raft_consensus.cc:374] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:37.125371 8199 raft_consensus.cc:729] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e6f0ab9db24450a82431ce46059fda, State: Initialized, Role: FOLLOWER
I20260925 13:07:37.125501 8198 ts_tablet_manager.cc:1428] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:07:37.125494 8199 consensus_queue.cc:260] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } } peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } }
I20260925 13:07:37.125546 7899 heartbeater.cc:503] Master 127.0.206.126:38937 was elected leader, sending a full tablet report...
I20260925 13:07:37.125737 8199 ts_tablet_manager.cc:1428] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 13:07:37.125789 8172 heartbeater.cc:503] Master 127.0.206.126:38937 was elected leader, sending a full tablet report...
I20260925 13:07:37.127236 8200 tablet_bootstrap.cc:492] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d: No bootstrap required, opened a new log
I20260925 13:07:37.127317 8200 ts_tablet_manager.cc:1397] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260925 13:07:37.128844 8200 raft_consensus.cc:348] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } } peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } }
I20260925 13:07:37.129007 8200 raft_consensus.cc:374] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:37.129052 8200 raft_consensus.cc:729] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1ea26f9b21b4a16af8b2434fd43cc7d, State: Initialized, Role: FOLLOWER
I20260925 13:07:37.129164 8200 consensus_queue.cc:260] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } } peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } }
I20260925 13:07:37.129422 8200 ts_tablet_manager.cc:1428] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 13:07:37.129463 8036 heartbeater.cc:503] Master 127.0.206.126:38937 was elected leader, sending a full tablet report...
I20260925 13:07:37.148097 8195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.145197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:36989 (local address 127.0.0.1:34738)
0925 13:07:37.145262 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:37.145274 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:37.145318 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:37.145558 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:37.145559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:37.145560 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:37.145638 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:37.145640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.146343 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.146344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:37.147189 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:37.147192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.147854 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.147855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:37.147915 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.147919 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:07:37.147963 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 13:07:37.148016 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 13:07:37.148097 8187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.145339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34738 (local address 127.0.206.124:36989)
0925 13:07:37.145420 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:37.145422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:37.145429 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:07:37.145453 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:37.145454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:37.145473 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:37.145554 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:37.145718 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.146246 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.147297 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.147762 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.147857 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.147980 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:37.147999 (+ 19us) server_negotiation.cc:299] Negotiation successful
0925 13:07:37.148037 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 13:07:37.148571 7725 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260925 13:07:37.149459 7759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b09bc81bfcd4976a0b4dc056fa50948. Latest consensus state: current_term: 1 leader_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: true } } }
I20260925 13:07:37.149561 7759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [sys.catalog]: This master's current role is: LEARNER
W20260925 13:07:37.198347 8039 tablet.cc:2369] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:07:37.199153 8206 raft_consensus.cc:482] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:37.199240 8206 raft_consensus.cc:504] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } } peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } }
I20260925 13:07:37.199645 8206 leader_election.cc:288] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e6f0ab9db24450a82431ce46059fda (127.0.206.67:37123), 5fc6c6780d8f4d14b8fe491aa3bc687a (127.0.206.65:45033)
I20260925 13:07:37.202561 8197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.199902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:33807 (local address 127.0.206.65:45033)
0925 13:07:37.199987 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:37.199989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:37.200001 (+ 12us) server_negotiation.cc:407] Connection header received
0925 13:07:37.200040 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:37.200041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:37.200050 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:37.200100 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:37.200305 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.200741 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.201784 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.202202 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.202280 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.202289 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:37.202461 (+ 172us) server_negotiation.cc:299] Negotiation successful
0925 13:07:37.202496 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 13:07:37.202566 8042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.199852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:45033 (local address 127.0.206.66:33807)
0925 13:07:37.199907 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:37.199917 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:37.199960 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:37.200124 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:37.200125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:37.200126 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:37.200198 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:37.200200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.200851 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.200852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:37.201687 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:37.201690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.202320 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.202321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:37.202383 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.202387 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:07:37.202456 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 13:07:37.202497 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":44}
I20260925 13:07:37.202622 8194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.199922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:41763 (local address 127.0.206.67:37123)
0925 13:07:37.199970 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:37.199973 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:37.199982 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:37.200010 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:37.200011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:37.200021 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:37.200100 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:37.200257 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.200733 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.201806 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.202261 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.202335 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.202345 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:37.202515 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 13:07:37.202553 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 13:07:37.202822 8040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.199827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:37123 (local address 127.0.206.66:41763)
0925 13:07:37.199884 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:37.199896 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:37.199940 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:37.200098 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:37.200098 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:37.200100 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:37.200184 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:37.200186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.200819 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.200820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:37.201690 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:37.201692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.202378 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.202379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:37.202434 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.202437 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:07:37.202510 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 13:07:37.202551 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":15}
I20260925 13:07:37.203128 8126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2ebed12913441149a6b003ef8108a47" candidate_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e6f0ab9db24450a82431ce46059fda" is_pre_election: true
I20260925 13:07:37.203317 8126 raft_consensus.cc:2454] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1ea26f9b21b4a16af8b2434fd43cc7d in term 0.
I20260925 13:07:37.203302 7853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2ebed12913441149a6b003ef8108a47" candidate_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" is_pre_election: true
I20260925 13:07:37.203428 7853 raft_consensus.cc:2454] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1ea26f9b21b4a16af8b2434fd43cc7d in term 0.
I20260925 13:07:37.203554 7926 leader_election.cc:302] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e6f0ab9db24450a82431ce46059fda, a1ea26f9b21b4a16af8b2434fd43cc7d; no voters:
I20260925 13:07:37.203801 8206 raft_consensus.cc:2781] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:07:37.203902 8206 raft_consensus.cc:482] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:07:37.203939 8206 raft_consensus.cc:3037] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:37.205180 8206 raft_consensus.cc:504] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } } peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } }
I20260925 13:07:37.205335 8206 leader_election.cc:288] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [CANDIDATE]: Term 1 election: Requested vote from peers 79e6f0ab9db24450a82431ce46059fda (127.0.206.67:37123), 5fc6c6780d8f4d14b8fe491aa3bc687a (127.0.206.65:45033)
I20260925 13:07:37.205470 7853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2ebed12913441149a6b003ef8108a47" candidate_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a"
I20260925 13:07:37.205530 8126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2ebed12913441149a6b003ef8108a47" candidate_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e6f0ab9db24450a82431ce46059fda"
I20260925 13:07:37.205544 7853 raft_consensus.cc:3037] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:37.205636 8126 raft_consensus.cc:3037] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:37.206315 7853 raft_consensus.cc:2454] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1ea26f9b21b4a16af8b2434fd43cc7d in term 1.
I20260925 13:07:37.206465 8126 raft_consensus.cc:2454] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1ea26f9b21b4a16af8b2434fd43cc7d in term 1.
I20260925 13:07:37.206517 7923 leader_election.cc:302] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fc6c6780d8f4d14b8fe491aa3bc687a, a1ea26f9b21b4a16af8b2434fd43cc7d; no voters:
I20260925 13:07:37.206631 8206 raft_consensus.cc:2781] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:37.206800 8206 raft_consensus.cc:686] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 1 LEADER]: Becoming Leader. State: Replica: a1ea26f9b21b4a16af8b2434fd43cc7d, State: Running, Role: LEADER
I20260925 13:07:37.206933 8206 consensus_queue.cc:237] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } } peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } }
I20260925 13:07:37.207684 6937 catalog_manager.cc:5495] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to a1ea26f9b21b4a16af8b2434fd43cc7d (127.0.206.66). New cstate: current_term: 1 leader_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } health_report { overall_health: UNKNOWN } } }
I20260925 13:07:37.208200 7496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: true } } }
I20260925 13:07:37.208303 7496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:37.209026 8213 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948: attempting to promote NON_VOTER 8895dce2e73c44d9be634ce437bfbaab to VOTER
I20260925 13:07:37.209148 8213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } }
I20260925 13:07:37.209573 7725 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 1 LEARNER]: Refusing update from remote peer 0b09bc81bfcd4976a0b4dc056fa50948: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260925 13:07:37.209692 7544 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 FOLLOWER]: Refusing update from remote peer 0b09bc81bfcd4976a0b4dc056fa50948: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260925 13:07:37.209794 7684 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260925 13:07:37.209887 8213 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260925 13:07:37.210572 8213 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 8895dce2e73c44d9be634ce437bfbaab (127.0.206.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } } }
I20260925 13:07:37.210722 7544 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 8895dce2e73c44d9be634ce437bfbaab (127.0.206.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } } }
I20260925 13:07:37.210749 7725 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 8895dce2e73c44d9be634ce437bfbaab (127.0.206.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } } }
I20260925 13:07:37.211454 8213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } } }
I20260925 13:07:37.211539 8213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:37.211505 7495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } } }
I20260925 13:07:37.211624 7495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:37.211602 8213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } } }
I20260925 13:07:37.211654 8213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:37.211675 7759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } } }
I20260925 13:07:37.211758 7759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:07:37.211997 8215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } } }
I20260925 13:07:37.212085 8215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:07:37.232957 8196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.230508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50246 (local address 127.0.206.66:32855)
0925 13:07:37.230546 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:37.230548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:37.230557 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:37.230628 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:37.230629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:37.230644 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:37.230695 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:37.230884 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.231381 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.232202 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.232452 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.232505 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.232650 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:37.232784 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:37.232849 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:37.232889 (+ 40us) server_negotiation.cc:299] Negotiation successful
0925 13:07:37.232904 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:07:37.234663 8126 raft_consensus.cc:1261] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda [term 1 FOLLOWER]: Refusing update from remote peer a1ea26f9b21b4a16af8b2434fd43cc7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:37.234683 7853 raft_consensus.cc:1261] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a [term 1 FOLLOWER]: Refusing update from remote peer a1ea26f9b21b4a16af8b2434fd43cc7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:37.235118 8211 consensus_queue.cc:1035] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:37.235257 8206 consensus_queue.cc:1035] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:37.238245 8217 mvcc.cc:205] Tried to move back new op lower bound from 7333239428030550016 to 7333239427920216064. Current Snapshot: MvccSnapshot[applied={T|T < 7333239428030550016}]
I20260925 13:07:37.238267 8219 mvcc.cc:205] Tried to move back new op lower bound from 7333239428030550016 to 7333239427920216064. Current Snapshot: MvccSnapshot[applied={T|T < 7333239428030550016}]
I20260925 13:07:37.238610 8218 mvcc.cc:205] Tried to move back new op lower bound from 7333239428030550016 to 7333239427920216064. Current Snapshot: MvccSnapshot[applied={T|T < 7333239428030550016}]
I20260925 13:07:37.241528 6937 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:37.241612 6937 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:37.243002 6937 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b09bc81bfcd4976a0b4dc056fa50948: Sending DeleteTablet for 3 replicas of tablet f2ebed12913441149a6b003ef8108a47
I20260925 13:07:37.243292 8106 tablet_service.cc:1514] Processing DeleteTablet for tablet f2ebed12913441149a6b003ef8108a47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:37 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:44278
I20260925 13:07:37.243321 7970 tablet_service.cc:1514] Processing DeleteTablet for tablet f2ebed12913441149a6b003ef8108a47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:37 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:50244
I20260925 13:07:37.243324 7833 tablet_service.cc:1514] Processing DeleteTablet for tablet f2ebed12913441149a6b003ef8108a47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:37 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:54352
I20260925 13:07:37.243541 8228 tablet_replica.cc:330] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d: stopping tablet replica
I20260925 13:07:37.243638 8228 raft_consensus.cc:2229] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:07:37.243719 8229 tablet_replica.cc:330] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a: stopping tablet replica
I20260925 13:07:37.243765 8228 raft_consensus.cc:2258] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:37.243795 8229 raft_consensus.cc:2229] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:37.243992 8227 tablet_replica.cc:330] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda: stopping tablet replica
I20260925 13:07:37.244077 8227 raft_consensus.cc:2229] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:37.244177 8227 raft_consensus.cc:2258] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:37.244292 8228 ts_tablet_manager.cc:1905] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:37.244573 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 6905
I20260925 13:07:37.244839 8227 ts_tablet_manager.cc:1905] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:37.246846 8227 ts_tablet_manager.cc:1918] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:07:37.246920 8227 log.cc:1196] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/f2ebed12913441149a6b003ef8108a47
I20260925 13:07:37.247174 8227 ts_tablet_manager.cc:1939] T f2ebed12913441149a6b003ef8108a47 P 79e6f0ab9db24450a82431ce46059fda: Deleting consensus metadata
I20260925 13:07:37.248529 8228 ts_tablet_manager.cc:1918] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:07:37.248607 8228 log.cc:1196] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/f2ebed12913441149a6b003ef8108a47
I20260925 13:07:37.248865 8228 ts_tablet_manager.cc:1939] T f2ebed12913441149a6b003ef8108a47 P a1ea26f9b21b4a16af8b2434fd43cc7d: Deleting consensus metadata
I20260925 13:07:37.249195 8229 raft_consensus.cc:2258] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:37.249663 8229 ts_tablet_manager.cc:1905] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:37.251731 8229 ts_tablet_manager.cc:1918] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:07:37.251857 8229 log.cc:1196] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-0/wal/wals/f2ebed12913441149a6b003ef8108a47
W20260925 13:07:37.252000 7925 connection.cc:535] server connection from 127.0.0.1:50244 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 13:07:37.252179 8062 connection.cc:535] server connection from 127.0.0.1:44278 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 13:07:37.252399 8229 ts_tablet_manager.cc:1939] T f2ebed12913441149a6b003ef8108a47 P 5fc6c6780d8f4d14b8fe491aa3bc687a: Deleting consensus metadata
W20260925 13:07:37.252769 7787 connection.cc:441] server connection from 127.0.0.1:54352 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54352 (request call id 1) could send its response
I20260925 13:07:37.257236 8187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.244045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34754 (local address 127.0.206.124:36989)
0925 13:07:37.244083 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:37.244086 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:37.249435 (+ 5349us) server_negotiation.cc:407] Connection header received
0925 13:07:37.249536 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:37.249537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:37.249548 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:37.249578 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:37.250344 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.251120 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.252221 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.252522 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.252566 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.254263 (+ 1697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:37.254264 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:37.254264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:37.254271 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:37.256015 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:37.256414 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:37.256416 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:37.256416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:37.256471 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:37.256663 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:37.256664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:37.256664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:37.256778 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:37.256848 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:37.257063 (+ 215us) server_negotiation.cc:299] Negotiation successful
0925 13:07:37.257142 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 13:07:37.261904 7906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.254159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.0.206.124:36989)
0925 13:07:37.254193 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:37.254194 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:37.254392 (+ 198us) server_negotiation.cc:407] Connection header received
0925 13:07:37.254499 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:37.254500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:37.254510 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:37.254539 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:37.255150 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.255832 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.256755 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.257033 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.257070 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.259164 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:37.259165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:37.259165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:37.259172 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:37.260800 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:37.261159 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:37.261160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:37.261160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:37.261215 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:37.261387 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:37.261388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:37.261388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:37.261502 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:37.261592 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:37.261723 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 13:07:37.261790 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 13:07:37.261967 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:37.253985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34778 (local address 127.0.206.125:43157)
0925 13:07:37.254021 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:37.254023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:37.254102 (+ 79us) server_negotiation.cc:407] Connection header received
0925 13:07:37.254200 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:37.254200 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:37.254213 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:37.254244 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:37.254920 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.255680 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.256693 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:37.257001 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:37.257041 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:37.258873 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:37.258875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:37.258875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:37.258882 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:37.260547 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:37.260938 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:37.260939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:37.260939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:37.260990 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:37.261195 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:37.261196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:37.261196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:37.261317 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:37.261411 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:37.261794 (+ 383us) server_negotiation.cc:299] Negotiation successful
0925 13:07:37.261877 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:07:38.239565 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:38.239261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.65:54213)
0925 13:07:38.239435 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:38.239510 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":91,"threads_started":1}
I20260925 13:07:38.239814 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:38.239526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.67:56283)
0925 13:07:38.239709 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:38.239774 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":49,"threads_started":1}
W20260925 13:07:38.239874 7899 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:38937 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
W20260925 13:07:38.239971 8172 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:38937 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
I20260925 13:07:38.240576 8239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:38.240346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.66:36533)
0925 13:07:38.240475 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:38.240538 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
W20260925 13:07:38.240753 8036 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:38937 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
I20260925 13:07:38.743305 8240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0b09bc81bfcd4976a0b4dc056fa50948)
I20260925 13:07:38.743420 8240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } }
I20260925 13:07:38.743505 8241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0b09bc81bfcd4976a0b4dc056fa50948)
I20260925 13:07:38.743633 8240 leader_election.cc:288] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0b09bc81bfcd4976a0b4dc056fa50948 (127.0.206.126:38937), 0ca60ab3dcb2456395272077367c4e44 (127.0.206.125:43157)
I20260925 13:07:38.743654 8241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } }
I20260925 13:07:38.743919 8241 leader_election.cc:288] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0b09bc81bfcd4976a0b4dc056fa50948 (127.0.206.126:38937), 8895dce2e73c44d9be634ce437bfbaab (127.0.206.124:36989)
I20260925 13:07:38.744113 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:38.743737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:34688)
0925 13:07:38.744013 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:38.744068 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":55,"threads_started":1}
I20260925 13:07:38.744200 8244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:38.743992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:34696)
0925 13:07:38.744108 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:38.744160 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":47,"threads_started":1}
W20260925 13:07:38.744244 7707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
W20260925 13:07:38.744481 7521 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
I20260925 13:07:38.744621 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:38.744533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:34704)
0925 13:07:38.744568 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:38.744591 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260925 13:07:38.744709 7707 leader_election.cc:334] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0b09bc81bfcd4976a0b4dc056fa50948 (127.0.206.126:38937): Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
I20260925 13:07:38.744895 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:38.744772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:34712)
0925 13:07:38.744815 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:38.744849 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260925 13:07:38.744993 7521 leader_election.cc:334] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0b09bc81bfcd4976a0b4dc056fa50948 (127.0.206.126:38937): Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
I20260925 13:07:38.747485 8244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:38.744125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:36989 (local address 127.0.0.1:34772)
0925 13:07:38.744270 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:38.744280 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:38.744330 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:38.744671 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:38.744672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:38.744675 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:38.744724 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:38.744725 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:38.745687 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:38.745688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:38.746417 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:38.746419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:38.747182 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:38.747183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:38.747251 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:38.747255 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:07:38.747335 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 13:07:38.747377 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":34,"threads_started":1}
I20260925 13:07:38.747493 8246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:38.744184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34794 (local address 127.0.206.125:43157)
0925 13:07:38.744286 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:38.744289 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:38.744312 (+ 23us) server_negotiation.cc:407] Connection header received
0925 13:07:38.744335 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:38.744336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:38.744345 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:38.744716 (+ 371us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:38.744744 (+ 28us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:38.745594 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:38.746573 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:38.747100 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:38.747192 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:38.747203 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:38.747351 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 13:07:38.747400 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":39,"threads_started":1}
I20260925 13:07:38.747519 8248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:38.744330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34772 (local address 127.0.206.124:36989)
0925 13:07:38.744544 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:38.744546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:38.744553 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:07:38.744571 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:38.744571 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:38.744581 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:38.744783 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:38.744812 (+ 29us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:38.745567 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:38.746534 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:38.747088 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:38.747182 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:38.747193 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:38.747342 (+ 149us) server_negotiation.cc:299] Negotiation successful
0925 13:07:38.747421 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":192,"thread_start_us":82,"threads_started":1}
I20260925 13:07:38.747522 8242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:38.743919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43157 (local address 127.0.0.1:34794)
0925 13:07:38.744017 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:38.744055 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:38.744124 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:38.744455 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:38.744456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:38.744458 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:38.744521 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:38.744523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:38.745675 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:38.745677 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:38.746458 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:38.746460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:38.747196 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:38.747196 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:38.747261 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:38.747264 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:07:38.747344 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 13:07:38.747414 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":16,"thread_start_us":44,"threads_started":1}
I20260925 13:07:38.747879 7544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8895dce2e73c44d9be634ce437bfbaab" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0ca60ab3dcb2456395272077367c4e44" is_pre_election: true
I20260925 13:07:38.747957 7544 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8895dce2e73c44d9be634ce437bfbaab in term 1.
I20260925 13:07:38.747969 7725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ca60ab3dcb2456395272077367c4e44" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8895dce2e73c44d9be634ce437bfbaab" is_pre_election: true
I20260925 13:07:38.748070 7725 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ca60ab3dcb2456395272077367c4e44 in term 1.
I20260925 13:07:38.748075 7707 leader_election.cc:302] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ca60ab3dcb2456395272077367c4e44, 8895dce2e73c44d9be634ce437bfbaab; no voters: 0b09bc81bfcd4976a0b4dc056fa50948
I20260925 13:07:38.748220 8240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 13:07:38.748222 7521 leader_election.cc:302] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ca60ab3dcb2456395272077367c4e44, 8895dce2e73c44d9be634ce437bfbaab; no voters: 0b09bc81bfcd4976a0b4dc056fa50948
I20260925 13:07:38.748255 8240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0b09bc81bfcd4976a0b4dc056fa50948)
I20260925 13:07:38.748270 8240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:07:38.748325 8241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 13:07:38.748358 8241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0b09bc81bfcd4976a0b4dc056fa50948)
I20260925 13:07:38.748376 8241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:07:38.748886 8240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } }
I20260925 13:07:38.749007 8240 leader_election.cc:288] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [CANDIDATE]: Term 2 election: Requested vote from peers 0b09bc81bfcd4976a0b4dc056fa50948 (127.0.206.126:38937), 0ca60ab3dcb2456395272077367c4e44 (127.0.206.125:43157)
I20260925 13:07:38.749125 8241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } }
I20260925 13:07:38.749233 8241 leader_election.cc:288] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [CANDIDATE]: Term 2 election: Requested vote from peers 0b09bc81bfcd4976a0b4dc056fa50948 (127.0.206.126:38937), 8895dce2e73c44d9be634ce437bfbaab (127.0.206.124:36989)
I20260925 13:07:38.749213 7544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8895dce2e73c44d9be634ce437bfbaab" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0ca60ab3dcb2456395272077367c4e44"
I20260925 13:07:38.749293 7544 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8895dce2e73c44d9be634ce437bfbaab in current term 2: Already voted for candidate 0ca60ab3dcb2456395272077367c4e44 in this term.
I20260925 13:07:38.749320 8242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:38.749080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:34722)
0925 13:07:38.749265 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:38.749294 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
I20260925 13:07:38.749368 8244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:38.749294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:34728)
0925 13:07:38.749324 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:38.749347 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260925 13:07:38.749449 7725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ca60ab3dcb2456395272077367c4e44" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8895dce2e73c44d9be634ce437bfbaab"
I20260925 13:07:38.749523 7725 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ca60ab3dcb2456395272077367c4e44 in current term 2: Already voted for candidate 8895dce2e73c44d9be634ce437bfbaab in this term.
I20260925 13:07:38.749632 8244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:38.749555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:34736)
0925 13:07:38.749587 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:38.749609 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260925 13:07:38.749709 7521 leader_election.cc:334] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0b09bc81bfcd4976a0b4dc056fa50948 (127.0.206.126:38937): Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
I20260925 13:07:38.749756 7521 leader_election.cc:302] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ca60ab3dcb2456395272077367c4e44; no voters: 0b09bc81bfcd4976a0b4dc056fa50948, 8895dce2e73c44d9be634ce437bfbaab
I20260925 13:07:38.749821 8242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:38.749751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:34742)
0925 13:07:38.749780 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:38.749803 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260925 13:07:38.749841 8241 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260925 13:07:38.749876 7707 leader_election.cc:334] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0b09bc81bfcd4976a0b4dc056fa50948 (127.0.206.126:38937): Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
I20260925 13:07:38.749919 7707 leader_election.cc:302] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8895dce2e73c44d9be634ce437bfbaab; no voters: 0b09bc81bfcd4976a0b4dc056fa50948, 0ca60ab3dcb2456395272077367c4e44
I20260925 13:07:38.749989 8240 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260925 13:07:39.240482 8251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:39.240238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.67:57199)
0925 13:07:39.240386 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:39.240426 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260925 13:07:39.240557 8250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:39.240236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.65:48345)
0925 13:07:39.240478 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:39.240514 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":55,"threads_started":1}
I20260925 13:07:39.241281 8252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:39.241074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.66:49283)
0925 13:07:39.241203 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:39.241238 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":50,"threads_started":1}
I20260925 13:07:40.241292 8253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.241027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.67:41287)
0925 13:07:40.241168 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:40.241211 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260925 13:07:40.241518 8254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.241324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.65:40317)
0925 13:07:40.241439 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:40.241473 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":48,"threads_started":1}
I20260925 13:07:40.241897 8255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.241695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.66:49313)
0925 13:07:40.241814 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:40.241850 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260925 13:07:40.771576 8256 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:40.771660 8256 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } }
I20260925 13:07:40.771842 8256 leader_election.cc:288] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0b09bc81bfcd4976a0b4dc056fa50948 (127.0.206.126:38937), 8895dce2e73c44d9be634ce437bfbaab (127.0.206.124:36989)
I20260925 13:07:40.772208 8257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.772016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:43836)
0925 13:07:40.772127 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:40.772168 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":67,"threads_started":1}
I20260925 13:07:40.772305 7725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ca60ab3dcb2456395272077367c4e44" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8895dce2e73c44d9be634ce437bfbaab" is_pre_election: true
I20260925 13:07:40.772418 7725 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ca60ab3dcb2456395272077367c4e44 in term 2.
I20260925 13:07:40.772611 7521 leader_election.cc:302] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ca60ab3dcb2456395272077367c4e44, 8895dce2e73c44d9be634ce437bfbaab; no voters:
I20260925 13:07:40.772708 8257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.772570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:43844)
0925 13:07:40.772634 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:40.772664 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 13:07:40.772734 8256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260925 13:07:40.772768 8256 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260925 13:07:40.772805 7521 leader_election.cc:334] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0b09bc81bfcd4976a0b4dc056fa50948 (127.0.206.126:38937): Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
I20260925 13:07:40.772810 8256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:07:40.773545 8256 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } }
I20260925 13:07:40.773684 8256 leader_election.cc:288] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [CANDIDATE]: Term 3 election: Requested vote from peers 0b09bc81bfcd4976a0b4dc056fa50948 (127.0.206.126:38937), 8895dce2e73c44d9be634ce437bfbaab (127.0.206.124:36989)
I20260925 13:07:40.773924 8257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.773788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:43850)
0925 13:07:40.773865 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:40.773897 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 13:07:40.773950 7725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ca60ab3dcb2456395272077367c4e44" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8895dce2e73c44d9be634ce437bfbaab"
I20260925 13:07:40.774060 7725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:07:40.774264 8257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.774165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:43858)
0925 13:07:40.774217 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:40.774243 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 13:07:40.774330 7521 leader_election.cc:334] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0b09bc81bfcd4976a0b4dc056fa50948 (127.0.206.126:38937): Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
I20260925 13:07:40.774847 7725 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ca60ab3dcb2456395272077367c4e44 in term 3.
I20260925 13:07:40.775023 7521 leader_election.cc:302] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ca60ab3dcb2456395272077367c4e44, 8895dce2e73c44d9be634ce437bfbaab; no voters: 0b09bc81bfcd4976a0b4dc056fa50948
I20260925 13:07:40.775125 8256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 13:07:40.775223 8256 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [term 3 LEADER]: Becoming Leader. State: Replica: 0ca60ab3dcb2456395272077367c4e44, State: Running, Role: LEADER
I20260925 13:07:40.775311 8256 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } }
I20260925 13:07:40.775657 8259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ca60ab3dcb2456395272077367c4e44. Latest consensus state: current_term: 3 leader_uuid: "0ca60ab3dcb2456395272077367c4e44" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } } }
I20260925 13:07:40.775758 8259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:40.775942 8261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:40.776304 8261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:07:40.776356 8261 catalog_manager.cc:1259] Loaded cluster ID: d4ab679f95aa42aaadb7f7d3c98d2d00
I20260925 13:07:40.776374 8261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:07:40.776505 8261 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:40.776556 8261 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: Loaded TSK: 0
I20260925 13:07:40.776643 8261 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:07:40.829026 7534 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:07:40.830534 7534 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:07:40.831666 7725 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [term 3 FOLLOWER]: Refusing update from remote peer 0ca60ab3dcb2456395272077367c4e44: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260925 13:07:40.831707 8257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.831582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:43872)
0925 13:07:40.831638 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:40.831667 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 13:07:40.832057 8257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.831966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.0.1:43888)
0925 13:07:40.831999 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:40.832021 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260925 13:07:40.832010 8256 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260925 13:07:40.832202 7521 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 -> Peer 0b09bc81bfcd4976a0b4dc056fa50948 (127.0.206.126:38937): Couldn't send request to peer 0b09bc81bfcd4976a0b4dc056fa50948. Status: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 13:07:40.832721 8263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ca60ab3dcb2456395272077367c4e44. Latest consensus state: current_term: 3 leader_uuid: "0ca60ab3dcb2456395272077367c4e44" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } } }
I20260925 13:07:40.832854 8263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:07:40.833256 8259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "0ca60ab3dcb2456395272077367c4e44" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } } }
I20260925 13:07:40.833339 8259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:40.833453 8263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "0ca60ab3dcb2456395272077367c4e44" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "0b09bc81bfcd4976a0b4dc056fa50948" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38937 } } peers { permanent_uuid: "0ca60ab3dcb2456395272077367c4e44" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43157 } attrs { promote: false } } peers { permanent_uuid: "8895dce2e73c44d9be634ce437bfbaab" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 36989 } attrs { promote: false } } }
I20260925 13:07:40.833536 8263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8895dce2e73c44d9be634ce437bfbaab [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:07:40.838191 8257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.835010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:45033 (local address 127.0.0.1:35942)
0925 13:07:40.835067 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:40.835077 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:40.835125 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:40.835384 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:40.835385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:40.835387 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:40.835438 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:40.835440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:40.836394 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:40.836394 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:40.837259 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:40.837261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:40.837971 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:40.837972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:40.838033 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:40.838036 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:07:40.838088 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 13:07:40.838129 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 13:07:40.838222 8269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.835094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35942 (local address 127.0.206.65:45033)
0925 13:07:40.835237 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:40.835239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:40.835247 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:40.835272 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:40.835273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:40.835285 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:40.835503 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:40.835539 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:40.836315 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:40.837375 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:40.837870 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:40.837968 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:40.837981 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:40.838108 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 13:07:40.838146 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":56,"threads_started":1}
I20260925 13:07:40.838415 7833 tablet_service.cc:1467] Processing CreateTablet for tablet b99d95984a764ffdad5b4163c3342c71 (DEFAULT_TABLE table=test-table [id=cbeb27f434d74c8fa8a5bf40c94948e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:40.838574 7833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99d95984a764ffdad5b4163c3342c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:40.838932 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.835122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:32855 (local address 127.0.0.1:58004)
0925 13:07:40.835228 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:40.835237 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:40.835276 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:40.835700 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:40.835700 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:40.835702 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:40.835742 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:40.835743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:40.836580 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:40.836580 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:40.837674 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:40.837676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:40.838692 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:40.838693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:40.838773 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:40.838776 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:07:40.838819 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 13:07:40.838864 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":67,"threads_started":1}
I20260925 13:07:40.838940 8271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.835174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58004 (local address 127.0.206.66:32855)
0925 13:07:40.835549 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:40.835551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:40.835558 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:07:40.835583 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:40.835585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:40.835596 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:40.835653 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:40.835829 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:40.836513 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:40.837755 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:40.838586 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:40.838652 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:40.838662 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:40.838838 (+ 176us) server_negotiation.cc:299] Negotiation successful
0925 13:07:40.838877 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":84,"threads_started":1}
I20260925 13:07:40.839135 7970 tablet_service.cc:1467] Processing CreateTablet for tablet b99d95984a764ffdad5b4163c3342c71 (DEFAULT_TABLE table=test-table [id=cbeb27f434d74c8fa8a5bf40c94948e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:40.839272 7970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99d95984a764ffdad5b4163c3342c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:40.839986 8272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.835279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:37123 (local address 127.0.0.1:40074)
0925 13:07:40.836431 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:40.836441 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:40.836488 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:40.836656 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:40.836657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:40.836659 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:40.836691 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:40.836693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:40.837591 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:40.837591 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:40.838662 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:40.838665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:40.839738 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:40.839739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:40.839803 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:40.839807 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:07:40.839873 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 13:07:40.839914 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1130,"thread_start_us":65,"threads_started":1}
I20260925 13:07:40.840279 8273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.835350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40074 (local address 127.0.206.67:37123)
0925 13:07:40.835460 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:40.835463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:40.836508 (+ 1045us) server_negotiation.cc:407] Connection header received
0925 13:07:40.836538 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:40.836539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:40.836549 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:40.836613 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:40.836774 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:40.837501 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:40.838768 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:40.839570 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:40.839649 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:40.839660 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:40.839898 (+ 238us) server_negotiation.cc:299] Negotiation successful
0925 13:07:40.839938 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":59,"threads_started":1}
I20260925 13:07:40.840471 8106 tablet_service.cc:1467] Processing CreateTablet for tablet b99d95984a764ffdad5b4163c3342c71 (DEFAULT_TABLE table=test-table [id=cbeb27f434d74c8fa8a5bf40c94948e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:40.840574 8106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99d95984a764ffdad5b4163c3342c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:40.840651 8275 tablet_bootstrap.cc:492] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d: Bootstrap starting.
I20260925 13:07:40.841009 8275 tablet_bootstrap.cc:654] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:40.841531 8275 tablet_bootstrap.cc:492] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d: No bootstrap required, opened a new log
I20260925 13:07:40.841572 8275 ts_tablet_manager.cc:1397] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:40.841683 8275 raft_consensus.cc:348] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } } peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } }
I20260925 13:07:40.841687 8277 tablet_bootstrap.cc:492] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda: Bootstrap starting.
I20260925 13:07:40.841749 8275 raft_consensus.cc:374] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:40.841773 8275 raft_consensus.cc:729] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1ea26f9b21b4a16af8b2434fd43cc7d, State: Initialized, Role: FOLLOWER
I20260925 13:07:40.841833 8275 consensus_queue.cc:260] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } } peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } }
I20260925 13:07:40.841866 8274 tablet_bootstrap.cc:492] T b99d95984a764ffdad5b4163c3342c71 P 5fc6c6780d8f4d14b8fe491aa3bc687a: Bootstrap starting.
I20260925 13:07:40.841964 8275 ts_tablet_manager.cc:1428] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:40.842084 8277 tablet_bootstrap.cc:654] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:40.842199 8274 tablet_bootstrap.cc:654] T b99d95984a764ffdad5b4163c3342c71 P 5fc6c6780d8f4d14b8fe491aa3bc687a: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:40.842406 8279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.842200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.66:56991)
0925 13:07:40.842328 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:40.842371 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
I20260925 13:07:40.842789 8279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.842640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.66:53507)
0925 13:07:40.842725 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:40.842760 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260925 13:07:40.842808 8038 heartbeater.cc:503] Master 127.0.206.125:43157 was elected leader, sending a full tablet report...
I20260925 13:07:40.843009 8277 tablet_bootstrap.cc:492] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda: No bootstrap required, opened a new log
I20260925 13:07:40.843062 8277 ts_tablet_manager.cc:1397] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:40.843176 8277 raft_consensus.cc:348] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } } peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } }
I20260925 13:07:40.843246 8277 raft_consensus.cc:374] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:40.843269 8277 raft_consensus.cc:729] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e6f0ab9db24450a82431ce46059fda, State: Initialized, Role: FOLLOWER
I20260925 13:07:40.843333 8277 consensus_queue.cc:260] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } } peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } }
I20260925 13:07:40.843504 8277 ts_tablet_manager.cc:1428] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:40.843528 8274 tablet_bootstrap.cc:492] T b99d95984a764ffdad5b4163c3342c71 P 5fc6c6780d8f4d14b8fe491aa3bc687a: No bootstrap required, opened a new log
I20260925 13:07:40.843624 8274 ts_tablet_manager.cc:1397] T b99d95984a764ffdad5b4163c3342c71 P 5fc6c6780d8f4d14b8fe491aa3bc687a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:07:40.843768 8274 raft_consensus.cc:348] T b99d95984a764ffdad5b4163c3342c71 P 5fc6c6780d8f4d14b8fe491aa3bc687a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } } peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } }
I20260925 13:07:40.843847 8274 raft_consensus.cc:374] T b99d95984a764ffdad5b4163c3342c71 P 5fc6c6780d8f4d14b8fe491aa3bc687a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:40.843874 8274 raft_consensus.cc:729] T b99d95984a764ffdad5b4163c3342c71 P 5fc6c6780d8f4d14b8fe491aa3bc687a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc6c6780d8f4d14b8fe491aa3bc687a, State: Initialized, Role: FOLLOWER
I20260925 13:07:40.843936 8274 consensus_queue.cc:260] T b99d95984a764ffdad5b4163c3342c71 P 5fc6c6780d8f4d14b8fe491aa3bc687a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } } peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } }
I20260925 13:07:40.844086 8274 ts_tablet_manager.cc:1428] T b99d95984a764ffdad5b4163c3342c71 P 5fc6c6780d8f4d14b8fe491aa3bc687a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:40.844007 8174 heartbeater.cc:503] Master 127.0.206.125:43157 was elected leader, sending a full tablet report...
I20260925 13:07:40.844426 8285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.844242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.65:33023)
0925 13:07:40.844356 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:40.844391 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":74,"threads_started":1}
I20260925 13:07:40.844688 8285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.844609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.65:44735)
0925 13:07:40.844642 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:40.844669 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260925 13:07:40.843804 8283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.843617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.67:48503)
0925 13:07:40.843728 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:40.843762 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":80,"threads_started":1}
I20260925 13:07:40.845026 8283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:40.844915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.67:40799)
0925 13:07:40.844976 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:40.845004 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 13:07:41.060122 8282 raft_consensus.cc:482] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:41.060221 8282 raft_consensus.cc:504] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } } peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } }
I20260925 13:07:41.060611 8282 leader_election.cc:288] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5fc6c6780d8f4d14b8fe491aa3bc687a (127.0.206.65:45033), a1ea26f9b21b4a16af8b2434fd43cc7d (127.0.206.66:32855)
I20260925 13:07:41.064502 8269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:41.060780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:43595 (local address 127.0.206.65:45033)
0925 13:07:41.060857 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:41.060859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:41.060871 (+ 12us) server_negotiation.cc:407] Connection header received
0925 13:07:41.060900 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:41.060901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:41.060912 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:41.060979 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:41.061178 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:41.061972 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:41.063548 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:41.064137 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:41.064200 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:41.064241 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:41.064406 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 13:07:41.064444 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 13:07:41.064502 8271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:41.061018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:45977 (local address 127.0.206.66:32855)
0925 13:07:41.061064 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:41.061066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:41.061073 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:07:41.061102 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:41.061103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:41.061109 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:41.061183 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:41.061325 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:41.062003 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:41.063548 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:41.064122 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:41.064213 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:41.064243 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:41.064390 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 13:07:41.064431 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 13:07:41.064502 8283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:41.060729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:45033 (local address 127.0.206.67:43595)
0925 13:07:41.060780 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:41.060791 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:41.060839 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:41.061009 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:41.061010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:41.061012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:41.061097 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:41.061099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:41.062084 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:41.062085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:41.063451 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:41.063454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:41.064217 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:41.064219 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:41.064297 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:41.064302 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 13:07:41.064357 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 13:07:41.064399 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":59}
I20260925 13:07:41.064517 8286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:41.060796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:32855 (local address 127.0.206.67:45977)
0925 13:07:41.060976 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:41.060986 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:41.061023 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:41.061203 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:41.061203 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:41.061204 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:41.061246 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:41.061251 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:41.062085 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:41.062085 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:41.063451 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:41.063454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:41.064234 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:41.064235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:41.064297 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:41.064301 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:07:41.064371 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 13:07:41.064399 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":82,"thread_start_us":119,"threads_started":1}
I20260925 13:07:41.064792 7853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99d95984a764ffdad5b4163c3342c71" candidate_uuid: "79e6f0ab9db24450a82431ce46059fda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" is_pre_election: true
I20260925 13:07:41.064917 7853 raft_consensus.cc:2454] T b99d95984a764ffdad5b4163c3342c71 P 5fc6c6780d8f4d14b8fe491aa3bc687a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e6f0ab9db24450a82431ce46059fda in term 0.
I20260925 13:07:41.064898 7990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99d95984a764ffdad5b4163c3342c71" candidate_uuid: "79e6f0ab9db24450a82431ce46059fda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" is_pre_election: true
I20260925 13:07:41.065024 7990 raft_consensus.cc:2454] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e6f0ab9db24450a82431ce46059fda in term 0.
I20260925 13:07:41.065110 8059 leader_election.cc:302] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fc6c6780d8f4d14b8fe491aa3bc687a, 79e6f0ab9db24450a82431ce46059fda; no voters:
I20260925 13:07:41.065234 8282 raft_consensus.cc:2781] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:07:41.065284 8282 raft_consensus.cc:482] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:07:41.065300 8282 raft_consensus.cc:3037] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:41.065999 8282 raft_consensus.cc:504] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } } peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } }
I20260925 13:07:41.066143 8282 leader_election.cc:288] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [CANDIDATE]: Term 1 election: Requested vote from peers 5fc6c6780d8f4d14b8fe491aa3bc687a (127.0.206.65:45033), a1ea26f9b21b4a16af8b2434fd43cc7d (127.0.206.66:32855)
I20260925 13:07:41.066296 7853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99d95984a764ffdad5b4163c3342c71" candidate_uuid: "79e6f0ab9db24450a82431ce46059fda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a"
I20260925 13:07:41.066332 7990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99d95984a764ffdad5b4163c3342c71" candidate_uuid: "79e6f0ab9db24450a82431ce46059fda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d"
I20260925 13:07:41.066393 7990 raft_consensus.cc:3037] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:41.066393 7853 raft_consensus.cc:3037] T b99d95984a764ffdad5b4163c3342c71 P 5fc6c6780d8f4d14b8fe491aa3bc687a [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:41.067157 7853 raft_consensus.cc:2454] T b99d95984a764ffdad5b4163c3342c71 P 5fc6c6780d8f4d14b8fe491aa3bc687a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e6f0ab9db24450a82431ce46059fda in term 1.
I20260925 13:07:41.067260 7990 raft_consensus.cc:2454] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e6f0ab9db24450a82431ce46059fda in term 1.
I20260925 13:07:41.067366 8059 leader_election.cc:302] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fc6c6780d8f4d14b8fe491aa3bc687a, 79e6f0ab9db24450a82431ce46059fda; no voters:
I20260925 13:07:41.067448 8282 raft_consensus.cc:2781] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:41.067581 8282 raft_consensus.cc:686] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [term 1 LEADER]: Becoming Leader. State: Replica: 79e6f0ab9db24450a82431ce46059fda, State: Running, Role: LEADER
I20260925 13:07:41.067636 8282 consensus_queue.cc:237] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } } peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } }
I20260925 13:07:41.068017 8286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:41.067863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.67:60901)
0925 13:07:41.067970 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:41.067995 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260925 13:07:41.068572 7534 catalog_manager.cc:5495] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda reported cstate change: term changed from 0 to 1, leader changed from <none> to 79e6f0ab9db24450a82431ce46059fda (127.0.206.67). New cstate: current_term: 1 leader_uuid: "79e6f0ab9db24450a82431ce46059fda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e6f0ab9db24450a82431ce46059fda" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37123 } health_report { overall_health: HEALTHY } } }
W20260925 13:07:41.070462 7902 tablet.cc:2369] T b99d95984a764ffdad5b4163c3342c71 P 5fc6c6780d8f4d14b8fe491aa3bc687a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:07:41.089885 8175 tablet.cc:2369] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:07:41.120764 8273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:41.118074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40086 (local address 127.0.206.67:37123)
0925 13:07:41.118142 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:41.118144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:41.118154 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:07:41.118262 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:41.118263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:41.118285 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:41.118379 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:41.118527 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:41.119105 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:41.119902 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:41.120113 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:41.120205 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:41.120380 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:41.120519 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:41.120615 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:41.120656 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 13:07:41.120677 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 13:07:41.121940 7853 raft_consensus.cc:1261] T b99d95984a764ffdad5b4163c3342c71 P 5fc6c6780d8f4d14b8fe491aa3bc687a [term 1 FOLLOWER]: Refusing update from remote peer 79e6f0ab9db24450a82431ce46059fda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:41.122021 7990 raft_consensus.cc:1261] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 1 FOLLOWER]: Refusing update from remote peer 79e6f0ab9db24450a82431ce46059fda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:41.122195 8287 consensus_queue.cc:1035] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fc6c6780d8f4d14b8fe491aa3bc687a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:41.122330 8282 consensus_queue.cc:1035] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1ea26f9b21b4a16af8b2434fd43cc7d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:41.122802 7901 heartbeater.cc:503] Master 127.0.206.125:43157 was elected leader, sending a full tablet report...
I20260925 13:07:41.123345 8285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:41.123198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.65:56021)
0925 13:07:41.123269 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:41.123308 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260925 13:07:41.123445 8279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:41.123297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.66:60635)
0925 13:07:41.123378 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:41.123413 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260925 13:07:41.124766 8286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:41.124611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.67:59617)
0925 13:07:41.124703 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:41.124729 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260925 13:07:41.125069 8279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:41.124983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.66:41685)
0925 13:07:41.125020 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:41.125043 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260925 13:07:41.125934 7534 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:41.126029 7534 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:41.126077 8286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:41.125766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.67:36601)
0925 13:07:41.126024 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:41.126052 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242}
I20260925 13:07:41.126641 8285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:41.125257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38937 (local address 127.0.206.65:40343)
0925 13:07:41.126582 (+ 1325us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:41.126614 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1248}
I20260925 13:07:41.127310 7534 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0ca60ab3dcb2456395272077367c4e44: Sending DeleteTablet for 3 replicas of tablet b99d95984a764ffdad5b4163c3342c71
I20260925 13:07:41.127643 7833 tablet_service.cc:1514] Processing DeleteTablet for tablet b99d95984a764ffdad5b4163c3342c71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:41 UTC) from {username='oryx', principal='oryx/127.0.206.125@KRBTEST.COM'} at 127.0.0.1:35942
I20260925 13:07:41.127705 8106 tablet_service.cc:1514] Processing DeleteTablet for tablet b99d95984a764ffdad5b4163c3342c71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:41 UTC) from {username='oryx', principal='oryx/127.0.206.125@KRBTEST.COM'} at 127.0.0.1:40074
I20260925 13:07:41.127732 7970 tablet_service.cc:1514] Processing DeleteTablet for tablet b99d95984a764ffdad5b4163c3342c71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:41 UTC) from {username='oryx', principal='oryx/127.0.206.125@KRBTEST.COM'} at 127.0.0.1:58004
I20260925 13:07:41.128046 8301 tablet_replica.cc:330] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d: stopping tablet replica
I20260925 13:07:41.128125 8301 raft_consensus.cc:2229] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:41.128136 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 7683
I20260925 13:07:41.128207 8301 raft_consensus.cc:2258] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:41.129037 8301 ts_tablet_manager.cc:1905] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:41.130225 8300 tablet_replica.cc:330] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda: stopping tablet replica
I20260925 13:07:41.130316 8300 raft_consensus.cc:2229] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:07:41.130407 8300 raft_consensus.cc:2258] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:41.130584 8301 ts_tablet_manager.cc:1918] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:07:41.130679 8301 log.cc:1196] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/b99d95984a764ffdad5b4163c3342c71
I20260925 13:07:41.130947 8301 ts_tablet_manager.cc:1939] T b99d95984a764ffdad5b4163c3342c71 P a1ea26f9b21b4a16af8b2434fd43cc7d: Deleting consensus metadata
I20260925 13:07:41.131289 7521 catalog_manager.cc:4841] TS a1ea26f9b21b4a16af8b2434fd43cc7d (127.0.206.66:32855): tablet b99d95984a764ffdad5b4163c3342c71 (table test-table [id=cbeb27f434d74c8fa8a5bf40c94948e9]) successfully deleted
I20260925 13:07:41.131448 8300 ts_tablet_manager.cc:1905] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:41.132866 8300 ts_tablet_manager.cc:1918] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:07:41.132937 8300 log.cc:1196] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/b99d95984a764ffdad5b4163c3342c71
I20260925 13:07:41.133158 8300 ts_tablet_manager.cc:1939] T b99d95984a764ffdad5b4163c3342c71 P 79e6f0ab9db24450a82431ce46059fda: Deleting consensus metadata
I20260925 13:07:41.133663 7521 catalog_manager.cc:4841] TS 79e6f0ab9db24450a82431ce46059fda (127.0.206.67:37123): tablet b99d95984a764ffdad5b4163c3342c71 (table test-table [id=cbeb27f434d74c8fa8a5bf40c94948e9]) successfully deleted
W20260925 13:07:41.136611 8272 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:41.136409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:45033 (local address 127.0.0.1:35954)
0925 13:07:41.136471 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:41.136481 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:41.136525 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:41.136573 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:45033: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 13:07:41.136780 7518 catalog_manager.cc:4584] TS 5fc6c6780d8f4d14b8fe491aa3bc687a (127.0.206.65:45033): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b99d95984a764ffdad5b4163c3342c71: Network error: Client connection negotiation failed: client connection to 127.0.206.65:45033: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260925 13:07:41.136826 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 7907
I20260925 13:07:41.144341 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 8043
I20260925 13:07:41.149822 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 7401
I20260925 13:07:41.154568 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 7588
2026-09-25T13:07:41Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6799 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:07:41 dist-test-slave-chkf krb5kdc[8304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:07:41 dist-test-slave-chkf krb5kdc[8304](info): set up 2 sockets
Sep 25 13:07:41 dist-test-slave-chkf krb5kdc[8304](info): commencing operation
krb5kdc: starting...
W20260925 13:07:42.199296 825 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:07:42 dist-test-slave-chkf krb5kdc[8304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:07:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/oryx_127.0.206.126.keytab.
Entry for principal oryx/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/oryx_127.0.206.126.keytab.
I20260925 13:07:42.318631 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:35803
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/oryx_127.0.206.126.keytab
--principal=oryx/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:35803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:42.398473 8317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:42.398682 8317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:42.398721 8317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:42.399854 8317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:42.399886 8317 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 13:07:42.399899 8317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:42.399912 8317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:42.399924 8317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:42.399936 8317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:07:42.401309 8317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:35803
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/oryx_127.0.206.126.keytab
--principal=oryx/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:35803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:42.401494 8317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:42.401660 8317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:42.406399 8323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:42.406412 8325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:42.406476 8317 server_base.cc:1034] running on GCE node
W20260925 13:07:42.406385 8322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:42.406924 8317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:42.407246 8317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:42.408421 8317 hybrid_clock.cc:648] HybridClock initialized: now 1790341662408409 us; error 54 us; skew 500 ppm
Sep 25 13:07:42 dist-test-slave-chkf krb5kdc[8304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341662, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:42.410907 8317 init.cc:377] Logged in from keytab as oryx/127.0.206.126@KRBTEST.COM (short username oryx)
I20260925 13:07:42.411284 8317 webserver.cc:460] Webserver started at http://127.0.206.126:39727/ using document root <none> and password file <none>
I20260925 13:07:42.411520 8317 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:42.411566 8317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:42.411720 8317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:42.412791 8317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "d6891f5172964bd8bb9acca9879009c3"
format_stamp: "Formatted at 2026-09-25 13:07:42 on dist-test-slave-chkf"
server_key: "442feeca4cb74eb73bde4e270159fed5"
server_key_iv: "0bfa2dc2c98f9964269e1c0a37e16d0c"
server_key_version: "encryptionkey@0"
I20260925 13:07:42.413228 8317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "d6891f5172964bd8bb9acca9879009c3"
format_stamp: "Formatted at 2026-09-25 13:07:42 on dist-test-slave-chkf"
server_key: "442feeca4cb74eb73bde4e270159fed5"
server_key_iv: "0bfa2dc2c98f9964269e1c0a37e16d0c"
server_key_version: "encryptionkey@0"
I20260925 13:07:42.414788 8317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:07:42.415602 8332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:42.415796 8317 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:42.415900 8317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "d6891f5172964bd8bb9acca9879009c3"
format_stamp: "Formatted at 2026-09-25 13:07:42 on dist-test-slave-chkf"
server_key: "442feeca4cb74eb73bde4e270159fed5"
server_key_iv: "0bfa2dc2c98f9964269e1c0a37e16d0c"
server_key_version: "encryptionkey@0"
I20260925 13:07:42.415994 8317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:42.430178 8317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:42.432179 8317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:42.432354 8317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:42.436975 8317 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:35803
I20260925 13:07:42.437094 8394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:35803 every 8 connection(s)
I20260925 13:07:42.437404 8317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:42.438084 8395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:42.440407 8395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3: Bootstrap starting.
I20260925 13:07:42.441002 8395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:42.441306 8395 log.cc:824] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:42.441972 8395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3: No bootstrap required, opened a new log
I20260925 13:07:42.443470 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 8317
I20260925 13:07:42.443559 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:07:42.443766 825 external_mini_cluster.cc:1428] Setting key 6e05c4e0669d649d11f4640d2b73d4ff
I20260925 13:07:42.444257 8395 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6891f5172964bd8bb9acca9879009c3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35803 } }
I20260925 13:07:42.444440 8395 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:42.444485 8395 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6891f5172964bd8bb9acca9879009c3, State: Initialized, Role: FOLLOWER
I20260925 13:07:42.444649 8395 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6891f5172964bd8bb9acca9879009c3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35803 } }
I20260925 13:07:42.444748 8395 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:42.444789 8395 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:42.444837 8395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:42.445354 8395 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6891f5172964bd8bb9acca9879009c3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35803 } }
I20260925 13:07:42.445473 8395 leader_election.cc:302] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6891f5172964bd8bb9acca9879009c3; no voters:
I20260925 13:07:42.445653 8395 leader_election.cc:288] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:07:42.445710 8400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:42.445868 8400 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3 [term 1 LEADER]: Becoming Leader. State: Replica: d6891f5172964bd8bb9acca9879009c3, State: Running, Role: LEADER
I20260925 13:07:42.445909 8395 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:42.445955 8400 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6891f5172964bd8bb9acca9879009c3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35803 } }
I20260925 13:07:42.446301 8402 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6891f5172964bd8bb9acca9879009c3. Latest consensus state: current_term: 1 leader_uuid: "d6891f5172964bd8bb9acca9879009c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6891f5172964bd8bb9acca9879009c3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35803 } } }
I20260925 13:07:42.446393 8402 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:42.446533 8401 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6891f5172964bd8bb9acca9879009c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6891f5172964bd8bb9acca9879009c3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35803 } } }
I20260925 13:07:42.446599 8401 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:42.446967 8407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:42.447495 8407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:07:42.449596 8407 catalog_manager.cc:1347] Generated new cluster ID: c3d105ee8c80473383468e9fc7ddeb42
I20260925 13:07:42.449644 8407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 13:07:42 dist-test-slave-chkf krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260925 13:07:42.454879 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.444303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44436 (local address 127.0.206.126:35803)
0925 13:07:42.444499 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:42.444509 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:42.444543 (+ 34us) server_negotiation.cc:407] Connection header received
0925 13:07:42.444680 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:42.444685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:42.444729 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:42.444879 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:42.445608 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.447490 (+ 1882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.448500 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.449070 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.449186 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.451644 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:42.451648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:42.451648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:42.451659 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:42.453343 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:42.453723 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:42.453726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:42.453727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:42.453830 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:42.454050 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:42.454051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:42.454052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:42.454281 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:42.454442 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:42.454543 (+ 101us) server_negotiation.cc:299] Negotiation successful
0925 13:07:42.454684 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":89,"threads_started":1}
I20260925 13:07:42.471007 8407 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:07:42.471747 8407 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:42.481011 8407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3: Generated new TSK 0
I20260925 13:07:42.481199 8407 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/oryx_127.0.206.65.keytab.
Entry for principal oryx/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/oryx_127.0.206.65.keytab.
I20260925 13:07:42.505826 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:35803
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/oryx_127.0.206.65.keytab
--principal=oryx/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:42.609115 8421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:42.609292 8421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:42.609315 8421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:42.610467 8421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:42.610507 8421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:42.610526 8421 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 13:07:42.610586 8421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:07:42.612066 8421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/oryx_127.0.206.65.keytab
--principal=oryx/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.206.126:35803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:42.612311 8421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:42.612533 8421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:42.617293 8427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:42.617314 8426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:42.617293 8429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:42.617489 8421 server_base.cc:1034] running on GCE node
I20260925 13:07:42.617745 8421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:42.617986 8421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:42.619160 8421 hybrid_clock.cc:648] HybridClock initialized: now 1790341662619146 us; error 28 us; skew 500 ppm
Sep 25 13:07:42 dist-test-slave-chkf krb5kdc[8304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341662, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:42.621546 8421 init.cc:377] Logged in from keytab as oryx/127.0.206.65@KRBTEST.COM (short username oryx)
I20260925 13:07:42.621989 8421 webserver.cc:460] Webserver started at http://127.0.206.65:40239/ using document root <none> and password file <none>
I20260925 13:07:42.622215 8421 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:42.622258 8421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:42.622402 8421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:42.623435 8421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "2f14f61c0a12443eb448bbcb217b5116"
format_stamp: "Formatted at 2026-09-25 13:07:42 on dist-test-slave-chkf"
server_key: "f0f002d51850e4cc8f0056227062626e"
server_key_iv: "dd5939729c8c57301d3e21f4b0c781ba"
server_key_version: "encryptionkey@0"
I20260925 13:07:42.623816 8421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "2f14f61c0a12443eb448bbcb217b5116"
format_stamp: "Formatted at 2026-09-25 13:07:42 on dist-test-slave-chkf"
server_key: "f0f002d51850e4cc8f0056227062626e"
server_key_iv: "dd5939729c8c57301d3e21f4b0c781ba"
server_key_version: "encryptionkey@0"
I20260925 13:07:42.625077 8421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 13:07:42.625735 8436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:42.625924 8421 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:07:42.625991 8421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "2f14f61c0a12443eb448bbcb217b5116"
format_stamp: "Formatted at 2026-09-25 13:07:42 on dist-test-slave-chkf"
server_key: "f0f002d51850e4cc8f0056227062626e"
server_key_iv: "dd5939729c8c57301d3e21f4b0c781ba"
server_key_version: "encryptionkey@0"
I20260925 13:07:42.626072 8421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:42.646832 8421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:42.648648 8421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:42.648798 8421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:42.649330 8421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:42.649386 8421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:42.649407 8421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:42.649441 8421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 13:07:42 dist-test-slave-chkf krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341662, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260925 13:07:42.656694 8421 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:45125
I20260925 13:07:42.656807 8551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:45125 every 8 connection(s)
I20260925 13:07:42.657073 8421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:07:42.659065 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.649701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:41715 (local address 127.0.206.126:35803)
0925 13:07:42.649759 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:42.649761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:42.649986 (+ 225us) server_negotiation.cc:407] Connection header received
0925 13:07:42.650474 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:42.650476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:42.650491 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:42.650553 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:42.651252 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.651769 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.652755 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.653045 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.653133 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.656279 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:42.656280 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:42.656281 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:42.656287 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:42.657830 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:42.658256 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:42.658257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:42.658257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:42.658306 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:42.658498 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:42.658498 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:42.658499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:42.658592 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:42.658639 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:42.658939 (+ 300us) server_negotiation.cc:299] Negotiation successful
0925 13:07:42.659001 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 13:07:42.659194 8445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.649702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:35803 (local address 127.0.206.65:41715)
0925 13:07:42.649862 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:42.649894 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:42.650367 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:42.650626 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:42.650629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:42.650978 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:42.651147 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:42.651152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.651870 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.651871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:42.652656 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:42.652658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.653141 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.653142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:42.653315 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.653831 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:42.653841 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:42.656165 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:42.657909 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:42.657937 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:42.657939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:42.658188 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:42.658365 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:42.658365 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:42.658366 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:42.658445 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:42.658648 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:42.658652 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:42.658814 (+ 162us) client_negotiation.cc:769] Sending connection context
0925 13:07:42.658901 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 13:07:42.659007 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
I20260925 13:07:42.659749 8552 heartbeater.cc:347] Connected to a master server at 127.0.206.126:35803
I20260925 13:07:42.659842 8552 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:42.660039 8552 heartbeater.cc:511] Master 127.0.206.126:35803 requested a full tablet report, sending...
I20260925 13:07:42.660519 8349 ts_manager.cc:194] Registered new tserver with Master: 2f14f61c0a12443eb448bbcb217b5116 (127.0.206.65:45125)
I20260925 13:07:42.661058 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 8421
I20260925 13:07:42.661188 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:07:42.661427 825 external_mini_cluster.cc:1428] Setting key dada28ff327acee6a52a7c085a484844
I20260925 13:07:42.661440 8349 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:41715
I20260925 13:07:42.666370 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.660731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:36477 (local address 127.0.206.126:35803)
0925 13:07:42.660768 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:42.660770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:42.660907 (+ 137us) server_negotiation.cc:407] Connection header received
0925 13:07:42.660935 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:42.660935 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:42.660945 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:42.661013 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:42.661322 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.662004 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.662751 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.663065 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.663137 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.663920 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:42.663922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:42.663922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:42.663928 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:42.665299 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:42.665642 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:42.665643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:42.665643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:42.665692 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:42.665892 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:42.665893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:42.665893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:42.665996 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:42.666069 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:42.666222 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 13:07:42.666289 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:07:42.666395 8558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.660684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:35803 (local address 127.0.206.65:36477)
0925 13:07:42.660818 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:42.660830 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:42.660892 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:42.661018 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:42.661018 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:42.661194 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:42.661253 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:42.661254 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.662075 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.662076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:42.662646 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:42.662648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.663139 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.663140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:42.663190 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.663581 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:42.663585 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:42.663833 (+ 248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:42.665392 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:42.665393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:42.665394 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:42.665558 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:42.665766 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:42.665766 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:42.665767 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:42.665820 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:42.666063 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:42.666064 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:42.666116 (+ 52us) client_negotiation.cc:769] Sending connection context
0925 13:07:42.666198 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 13:07:42.666289 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for oryx/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/oryx_127.0.206.66.keytab.
Entry for principal oryx/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/oryx_127.0.206.66.keytab.
I20260925 13:07:42.686252 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:35803
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/oryx_127.0.206.66.keytab
--principal=oryx/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:42.764047 8561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:42.764219 8561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:42.764242 8561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:42.765281 8561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:42.765313 8561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:42.765333 8561 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 13:07:42.765350 8561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:07:42.766681 8561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/oryx_127.0.206.66.keytab
--principal=oryx/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.206.126:35803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:42.766865 8561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:42.767019 8561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:42.771793 8567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:42.771785 8566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:42.771785 8569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:42.771936 8561 server_base.cc:1034] running on GCE node
I20260925 13:07:42.772181 8561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:42.772372 8561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:42.773510 8561 hybrid_clock.cc:648] HybridClock initialized: now 1790341662773488 us; error 38 us; skew 500 ppm
Sep 25 13:07:42 dist-test-slave-chkf krb5kdc[8304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341662, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:42.775909 8561 init.cc:377] Logged in from keytab as oryx/127.0.206.66@KRBTEST.COM (short username oryx)
I20260925 13:07:42.776330 8561 webserver.cc:460] Webserver started at http://127.0.206.66:33181/ using document root <none> and password file <none>
I20260925 13:07:42.776562 8561 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:42.776608 8561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:42.776690 8561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:42.777580 8561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "0f330d8bdff34b24804edef4ea2f5121"
format_stamp: "Formatted at 2026-09-25 13:07:42 on dist-test-slave-chkf"
server_key: "f5bea238eac6cf26d2f35ecc6d677d02"
server_key_iv: "126f2f42107fc6e889ec0a0a937dfe78"
server_key_version: "encryptionkey@0"
I20260925 13:07:42.777855 8561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "0f330d8bdff34b24804edef4ea2f5121"
format_stamp: "Formatted at 2026-09-25 13:07:42 on dist-test-slave-chkf"
server_key: "f5bea238eac6cf26d2f35ecc6d677d02"
server_key_iv: "126f2f42107fc6e889ec0a0a937dfe78"
server_key_version: "encryptionkey@0"
I20260925 13:07:42.779091 8561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:07:42.779731 8576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:42.779914 8561 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:42.780014 8561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "0f330d8bdff34b24804edef4ea2f5121"
format_stamp: "Formatted at 2026-09-25 13:07:42 on dist-test-slave-chkf"
server_key: "f5bea238eac6cf26d2f35ecc6d677d02"
server_key_iv: "126f2f42107fc6e889ec0a0a937dfe78"
server_key_version: "encryptionkey@0"
I20260925 13:07:42.780098 8561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:42.798377 8561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:42.800078 8561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:42.800238 8561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:42.800738 8561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:42.800798 8561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:42.800830 8561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:42.800869 8561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 13:07:42 dist-test-slave-chkf krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341662, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260925 13:07:42.807996 8561 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:42371
I20260925 13:07:42.808364 8561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:07:42.808786 8691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:42371 every 8 connection(s)
I20260925 13:07:42.810179 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.801071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:48447 (local address 127.0.206.126:35803)
0925 13:07:42.801122 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:42.801125 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:42.801366 (+ 241us) server_negotiation.cc:407] Connection header received
0925 13:07:42.801776 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:42.801777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:42.801792 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:42.801872 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:42.802613 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.803280 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.804314 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.804522 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.804607 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.807578 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:42.807580 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:42.807580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:42.807588 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:42.809138 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:42.809478 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:42.809478 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:42.809479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:42.809525 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:42.809722 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:42.809723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:42.809723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:42.809815 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:42.809863 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:42.810066 (+ 203us) server_negotiation.cc:299] Negotiation successful
0925 13:07:42.810117 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 13:07:42.810403 8585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.801076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:35803 (local address 127.0.206.66:48447)
0925 13:07:42.801227 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:42.801264 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:42.801680 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:42.801916 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:42.801919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:42.802226 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:42.802503 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:42.802509 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.803376 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.803377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:42.804205 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:42.804208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.804616 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.804618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:42.804739 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.805250 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:42.805261 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:42.807486 (+ 2225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:42.809203 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:42.809207 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:42.809208 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:42.809411 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:42.809580 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:42.809581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:42.809581 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:42.809669 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:42.809877 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:42.809880 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:42.809977 (+ 97us) client_negotiation.cc:769] Sending connection context
0925 13:07:42.810053 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 13:07:42.810154 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260925 13:07:42.810508 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 8561
I20260925 13:07:42.810612 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:07:42.810834 825 external_mini_cluster.cc:1428] Setting key df948812c0ece50cf8d974e6474d5728
I20260925 13:07:42.810866 8692 heartbeater.cc:347] Connected to a master server at 127.0.206.126:35803
I20260925 13:07:42.810941 8692 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:42.811120 8692 heartbeater.cc:511] Master 127.0.206.126:35803 requested a full tablet report, sending...
I20260925 13:07:42.811477 8349 ts_manager.cc:194] Registered new tserver with Master: 0f330d8bdff34b24804edef4ea2f5121 (127.0.206.66:42371)
I20260925 13:07:42.811836 8349 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:48447
I20260925 13:07:42.818329 8698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.812158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:35803 (local address 127.0.206.66:42681)
0925 13:07:42.812282 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:42.812292 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:42.812342 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:42.812456 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:42.812457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:42.812629 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:42.812676 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:42.812677 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.813568 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.813569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:42.814215 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:42.814217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.814829 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.814830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:42.814934 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.815371 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:42.815376 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:42.815643 (+ 267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:42.817331 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:42.817332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:42.817332 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:42.817483 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:42.817692 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:42.817693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:42.817693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:42.817739 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:42.818086 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:42.818087 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:42.818121 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 13:07:42.818173 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 13:07:42.818242 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":68,"threads_started":1}
I20260925 13:07:42.818404 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.812208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:42681 (local address 127.0.206.126:35803)
0925 13:07:42.812254 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:42.812257 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:42.812329 (+ 72us) server_negotiation.cc:407] Connection header received
0925 13:07:42.812388 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:42.812389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:42.812399 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:42.812452 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:42.812780 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.813460 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.814313 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.814627 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.814743 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.815745 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:42.815747 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:42.815747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:42.815754 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:42.817251 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:42.817561 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:42.817562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:42.817562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:42.817625 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:42.817807 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:42.817808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:42.817808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:42.818019 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:42.818089 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:42.818202 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 13:07:42.818269 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for oryx/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/oryx_127.0.206.67.keytab.
Entry for principal oryx/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/oryx_127.0.206.67.keytab.
I20260925 13:07:42.837204 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:35803
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/oryx_127.0.206.67.keytab
--principal=oryx/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:42.914477 8701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:42.914687 8701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:42.914722 8701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:42.915970 8701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:42.916036 8701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:42.916072 8701 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 13:07:42.916106 8701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:07:42.917456 8701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/krb5kdc/oryx_127.0.206.67.keytab
--principal=oryx/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.206.126:35803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:42.917704 8701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:42.917940 8701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:42.922691 8709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:42.922732 8706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:42.922768 8707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:42.922842 8701 server_base.cc:1034] running on GCE node
I20260925 13:07:42.923116 8701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:42.923393 8701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:42.924556 8701 hybrid_clock.cc:648] HybridClock initialized: now 1790341662924528 us; error 39 us; skew 500 ppm
Sep 25 13:07:42 dist-test-slave-chkf krb5kdc[8304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341662, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:42.926940 8701 init.cc:377] Logged in from keytab as oryx/127.0.206.67@KRBTEST.COM (short username oryx)
I20260925 13:07:42.927290 8701 webserver.cc:460] Webserver started at http://127.0.206.67:39643/ using document root <none> and password file <none>
I20260925 13:07:42.927456 8701 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:42.927497 8701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:42.927577 8701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:42.928493 8701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "b540febd45d340fd86f9f8cf6f1b57db"
format_stamp: "Formatted at 2026-09-25 13:07:42 on dist-test-slave-chkf"
server_key: "c50fffa5e1714b10d385fabbb13b9140"
server_key_iv: "db97cdb011ca2bcfa76fd044f577775b"
server_key_version: "encryptionkey@0"
I20260925 13:07:42.928857 8701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "b540febd45d340fd86f9f8cf6f1b57db"
format_stamp: "Formatted at 2026-09-25 13:07:42 on dist-test-slave-chkf"
server_key: "c50fffa5e1714b10d385fabbb13b9140"
server_key_iv: "db97cdb011ca2bcfa76fd044f577775b"
server_key_version: "encryptionkey@0"
I20260925 13:07:42.930087 8701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 13:07:42.930756 8716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:42.930936 8701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:42.931001 8701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "b540febd45d340fd86f9f8cf6f1b57db"
format_stamp: "Formatted at 2026-09-25 13:07:42 on dist-test-slave-chkf"
server_key: "c50fffa5e1714b10d385fabbb13b9140"
server_key_iv: "db97cdb011ca2bcfa76fd044f577775b"
server_key_version: "encryptionkey@0"
I20260925 13:07:42.931047 8701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:42.944631 8701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:42.946427 8701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:42.946525 8701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:42.947119 8701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:42.947163 8701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:42.947207 8701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:42.947237 8701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 13:07:42 dist-test-slave-chkf krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341662, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260925 13:07:42.954101 8701 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:33689
I20260925 13:07:42.954162 8831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:33689 every 8 connection(s)
I20260925 13:07:42.954432 8701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:07:42.957067 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.947558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:57137 (local address 127.0.206.126:35803)
0925 13:07:42.947629 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:42.947632 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:42.947784 (+ 152us) server_negotiation.cc:407] Connection header received
0925 13:07:42.948246 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:42.948248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:42.948260 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:42.948309 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:42.949069 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.949542 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.951061 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.951378 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.951446 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.954112 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:42.954114 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:42.954114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:42.954121 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:42.955681 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:42.956092 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:42.956093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:42.956094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:42.956148 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:42.956407 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:42.956408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:42.956409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:42.956522 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:42.956599 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:42.956854 (+ 255us) server_negotiation.cc:299] Negotiation successful
0925 13:07:42.956934 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 13:07:42.957201 8725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.947535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:35803 (local address 127.0.206.67:57137)
0925 13:07:42.947665 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:42.947700 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:42.948138 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:42.948367 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:42.948371 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:42.948701 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:42.948967 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:42.948972 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.949635 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.949636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:42.950981 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:42.950985 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.951485 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.951486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:42.951652 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.952214 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:42.952223 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:42.954000 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:42.955782 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:42.955788 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:42.955790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:42.955998 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:42.956225 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:42.956226 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:42.956226 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:42.956334 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:42.956591 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:42.956596 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:42.956726 (+ 130us) client_negotiation.cc:769] Sending connection context
0925 13:07:42.956833 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 13:07:42.956969 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":70,"threads_started":1}
I20260925 13:07:42.957685 8832 heartbeater.cc:347] Connected to a master server at 127.0.206.126:35803
I20260925 13:07:42.957813 8832 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:42.957971 8832 heartbeater.cc:511] Master 127.0.206.126:35803 requested a full tablet report, sending...
I20260925 13:07:42.958328 8348 ts_manager.cc:194] Registered new tserver with Master: b540febd45d340fd86f9f8cf6f1b57db (127.0.206.67:33689)
I20260925 13:07:42.958853 8348 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:57137
I20260925 13:07:42.961164 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 8701
I20260925 13:07:42.961279 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:07:42.961460 825 external_mini_cluster.cc:1428] Setting key ef25d58fcb5b613af9afd0919b11bb6a
I20260925 13:07:42.962314 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:07:42.964155 8838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.958600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:35803 (local address 127.0.206.67:48505)
0925 13:07:42.958746 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:42.958756 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:42.958819 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:42.958957 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:42.958958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:42.959122 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:42.959170 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:42.959171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.959789 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.959790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:42.960344 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:42.960346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.960715 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.960715 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:42.960780 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.961239 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:42.961243 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:42.961578 (+ 335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:42.963359 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:42.963360 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:42.963361 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:42.963509 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:42.963693 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:42.963693 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:42.963693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:42.963732 (+ 39us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:42.963950 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:42.963951 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:42.963983 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 13:07:42.964034 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 13:07:42.964098 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260925 13:07:42.964301 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.958688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:48505 (local address 127.0.206.126:35803)
0925 13:07:42.958751 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:42.958753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:42.958808 (+ 55us) server_negotiation.cc:407] Connection header received
0925 13:07:42.958874 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:42.958874 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:42.958881 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:42.958953 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:42.959249 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.959697 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.960430 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.960628 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.960712 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.961684 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:42.961685 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:42.961686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:42.961692 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:42.963279 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:42.963579 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:42.963580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:42.963580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:42.963636 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:42.963767 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:42.963768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:42.963768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:42.963890 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:42.963949 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:42.964084 (+ 135us) server_negotiation.cc:299] Negotiation successful
0925 13:07:42.964159 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Sep 25 13:07:42 dist-test-slave-chkf krb5kdc[8304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341662, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 13:07:42 dist-test-slave-chkf krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:42 dist-test-slave-chkf krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 13:07:42.983474 8399 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:42.978388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44442 (local address 127.0.206.126:35803)
0925 13:07:42.978441 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:42.978443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:42.978468 (+ 25us) server_negotiation.cc:407] Connection header received
0925 13:07:42.978485 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:42.978486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:42.978493 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:42.978531 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:42.979130 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.979566 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.980260 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.980427 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.980472 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.983377 (+ 2905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44442: BlockingRecv error: recv got EOF from 127.0.0.1:44442 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 25 13:07:42 dist-test-slave-chkf krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341662, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260925 13:07:42.990337 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.984213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44448 (local address 127.0.206.126:35803)
0925 13:07:42.984287 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:42.984289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:42.984324 (+ 35us) server_negotiation.cc:407] Connection header received
0925 13:07:42.984395 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:42.984396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:42.984405 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:42.984447 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:42.984915 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.985374 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.986035 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.986232 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.986324 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.988239 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:42.988240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:42.988241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:42.988246 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:42.989549 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:42.989835 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:42.989836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:42.989836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:42.989883 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:42.990023 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:42.990023 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:42.990024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:42.990100 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:42.990159 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:42.990240 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 13:07:42.990286 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 13:07:42.991830 8348 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:07:42.993288 8348 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:07:42.999406 8855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.995631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:42371 (local address 127.0.0.1:34746)
0925 13:07:42.995810 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:42.995819 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:42.995856 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:42.996187 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:42.996188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:42.996190 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:42.996229 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:42.996230 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.997410 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.997411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:42.998329 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:42.998332 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.999120 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.999122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:42.999197 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.999211 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 13:07:42.999276 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 13:07:42.999323 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":58,"threads_started":1}
I20260925 13:07:42.999507 8858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.995457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:33689 (local address 127.0.0.1:49924)
0925 13:07:42.995723 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:42.995734 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:42.995781 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:42.996151 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:42.996152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:42.996153 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:42.996197 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:42.996198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.997034 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.997035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:42.998343 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:42.998345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.999257 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.999258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:42.999319 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.999321 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:07:42.999375 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 13:07:42.999405 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":80,"threads_started":1}
I20260925 13:07:42.999591 8854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.995366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:45125 (local address 127.0.0.1:43212)
0925 13:07:42.995558 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:42.995595 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:42.995657 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:42.995939 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:42.995943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:42.995944 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:42.996003 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:42.996005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.997251 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.997252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:42.998343 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:42.998345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.999347 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.999348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:42.999428 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.999432 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:07:42.999481 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 13:07:42.999531 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":26,"thread_start_us":82,"threads_started":1}
I20260925 13:07:42.999715 8859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.995822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34746 (local address 127.0.206.66:42371)
0925 13:07:42.995988 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:42.995992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:42.996000 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:42.996035 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:42.996037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:42.996053 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:42.996125 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:42.996316 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.997329 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.998412 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.999019 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.999117 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.999189 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:42.999581 (+ 392us) server_negotiation.cc:299] Negotiation successful
0925 13:07:42.999625 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":69,"threads_started":1}
I20260925 13:07:42.999878 8857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.995567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43212 (local address 127.0.206.65:45125)
0925 13:07:42.995753 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:42.995757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:42.995767 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:07:42.995814 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:42.995816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:42.995830 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:42.995930 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:42.996077 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.997157 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.998444 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.999180 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.999247 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.999314 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:42.999748 (+ 434us) server_negotiation.cc:299] Negotiation successful
0925 13:07:42.999785 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":83,"threads_started":1}
I20260925 13:07:42.999959 8856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:42.995534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49924 (local address 127.0.206.67:33689)
0925 13:07:42.995959 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:42.995964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:42.995972 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:42.996006 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:42.996008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:42.996023 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:42.996095 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:42.996269 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.996945 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.998444 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:42.999183 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:42.999244 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:42.999312 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:42.999389 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 13:07:42.999425 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":71,"threads_started":1}
I20260925 13:07:43.000437 8626 tablet_service.cc:1467] Processing CreateTablet for tablet def14d7fd3bb46b690eca4bb54a1da52 (DEFAULT_TABLE table=test-table [id=6731887352104e85a2b3bfae4bdd4293]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:43.000444 8486 tablet_service.cc:1467] Processing CreateTablet for tablet def14d7fd3bb46b690eca4bb54a1da52 (DEFAULT_TABLE table=test-table [id=6731887352104e85a2b3bfae4bdd4293]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:43.000444 8766 tablet_service.cc:1467] Processing CreateTablet for tablet def14d7fd3bb46b690eca4bb54a1da52 (DEFAULT_TABLE table=test-table [id=6731887352104e85a2b3bfae4bdd4293]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:43.000751 8626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet def14d7fd3bb46b690eca4bb54a1da52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:43.000749 8766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet def14d7fd3bb46b690eca4bb54a1da52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:43.000749 8486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet def14d7fd3bb46b690eca4bb54a1da52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:43.003755 8860 tablet_bootstrap.cc:492] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121: Bootstrap starting.
I20260925 13:07:43.004364 8861 tablet_bootstrap.cc:492] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db: Bootstrap starting.
I20260925 13:07:43.004509 8860 tablet_bootstrap.cc:654] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:43.004654 8862 tablet_bootstrap.cc:492] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116: Bootstrap starting.
I20260925 13:07:43.004860 8860 log.cc:824] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:43.004951 8861 tablet_bootstrap.cc:654] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:43.005270 8862 tablet_bootstrap.cc:654] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:43.005272 8861 log.cc:824] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:43.005601 8862 log.cc:824] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:43.005697 8860 tablet_bootstrap.cc:492] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121: No bootstrap required, opened a new log
I20260925 13:07:43.005776 8860 ts_tablet_manager.cc:1397] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:07:43.005973 8861 tablet_bootstrap.cc:492] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db: No bootstrap required, opened a new log
I20260925 13:07:43.006042 8861 ts_tablet_manager.cc:1397] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:07:43.006321 8862 tablet_bootstrap.cc:492] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116: No bootstrap required, opened a new log
I20260925 13:07:43.006407 8862 ts_tablet_manager.cc:1397] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:07:43.007764 8860 raft_consensus.cc:348] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } }
I20260925 13:07:43.007946 8860 raft_consensus.cc:374] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:43.007838 8862 raft_consensus.cc:348] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } }
I20260925 13:07:43.007980 8860 raft_consensus.cc:729] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f330d8bdff34b24804edef4ea2f5121, State: Initialized, Role: FOLLOWER
I20260925 13:07:43.007984 8862 raft_consensus.cc:374] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:43.008025 8862 raft_consensus.cc:729] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f14f61c0a12443eb448bbcb217b5116, State: Initialized, Role: FOLLOWER
I20260925 13:07:43.008033 8861 raft_consensus.cc:348] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } }
I20260925 13:07:43.008112 8860 consensus_queue.cc:260] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } }
I20260925 13:07:43.008170 8861 raft_consensus.cc:374] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:43.008140 8862 consensus_queue.cc:260] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } }
I20260925 13:07:43.008211 8861 raft_consensus.cc:729] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b540febd45d340fd86f9f8cf6f1b57db, State: Initialized, Role: FOLLOWER
I20260925 13:07:43.008363 8860 ts_tablet_manager.cc:1428] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 13:07:43.008342 8861 consensus_queue.cc:260] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } }
I20260925 13:07:43.008401 8552 heartbeater.cc:503] Master 127.0.206.126:35803 was elected leader, sending a full tablet report...
I20260925 13:07:43.008474 8692 heartbeater.cc:503] Master 127.0.206.126:35803 was elected leader, sending a full tablet report...
I20260925 13:07:43.008574 8861 ts_tablet_manager.cc:1428] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 13:07:43.008638 8862 ts_tablet_manager.cc:1428] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 13:07:43.008702 8832 heartbeater.cc:503] Master 127.0.206.126:35803 was elected leader, sending a full tablet report...
W20260925 13:07:43.058877 8693 tablet.cc:2369] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:07:43.134145 8867 raft_consensus.cc:482] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:43.134267 8867 raft_consensus.cc:504] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } }
I20260925 13:07:43.134781 8867 leader_election.cc:288] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b540febd45d340fd86f9f8cf6f1b57db (127.0.206.67:33689), 0f330d8bdff34b24804edef4ea2f5121 (127.0.206.66:42371)
I20260925 13:07:43.138654 8856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.135079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:44911 (local address 127.0.206.67:33689)
0925 13:07:43.135128 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:43.135130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:43.135142 (+ 12us) server_negotiation.cc:407] Connection header received
0925 13:07:43.135174 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:43.135175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:43.135185 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:43.135250 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:43.135416 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.136151 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.137726 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.138284 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.138370 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.138379 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:43.138547 (+ 168us) server_negotiation.cc:299] Negotiation successful
0925 13:07:43.138586 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 13:07:43.138654 8859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.135184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:37293 (local address 127.0.206.66:42371)
0925 13:07:43.135232 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:43.135234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:43.135267 (+ 33us) server_negotiation.cc:407] Connection header received
0925 13:07:43.135322 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:43.135323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:43.135332 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:43.135417 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:43.135524 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.136206 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.137727 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.138284 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.138362 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.138371 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:43.138548 (+ 177us) server_negotiation.cc:299] Negotiation successful
0925 13:07:43.138586 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 13:07:43.138696 8869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.135080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:42371 (local address 127.0.206.65:37293)
0925 13:07:43.135218 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:43.135227 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:43.135273 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:43.135422 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:43.135423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:43.135424 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:43.135480 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.135480 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.136296 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.136297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.137625 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.137627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.138377 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.138378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.138463 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.138466 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:07:43.138527 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 13:07:43.138578 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":50,"thread_start_us":77,"threads_started":1}
I20260925 13:07:43.138731 8445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.134956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:33689 (local address 127.0.206.65:44911)
0925 13:07:43.135018 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:43.135031 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:43.135119 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:43.135258 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:43.135259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:43.135261 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:43.135340 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.135342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.136287 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.136288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.137625 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.137627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.138377 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.138378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.138463 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.138466 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:07:43.138527 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 13:07:43.138578 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":71}
I20260925 13:07:43.139274 8786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "def14d7fd3bb46b690eca4bb54a1da52" candidate_uuid: "2f14f61c0a12443eb448bbcb217b5116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b540febd45d340fd86f9f8cf6f1b57db" is_pre_election: true
I20260925 13:07:43.139309 8646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "def14d7fd3bb46b690eca4bb54a1da52" candidate_uuid: "2f14f61c0a12443eb448bbcb217b5116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f330d8bdff34b24804edef4ea2f5121" is_pre_election: true
I20260925 13:07:43.139431 8786 raft_consensus.cc:2454] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f14f61c0a12443eb448bbcb217b5116 in term 0.
I20260925 13:07:43.139431 8646 raft_consensus.cc:2454] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f14f61c0a12443eb448bbcb217b5116 in term 0.
I20260925 13:07:43.139637 8440 leader_election.cc:302] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f330d8bdff34b24804edef4ea2f5121, 2f14f61c0a12443eb448bbcb217b5116; no voters:
I20260925 13:07:43.139775 8867 raft_consensus.cc:2781] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:07:43.139829 8867 raft_consensus.cc:482] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:07:43.139853 8867 raft_consensus.cc:3037] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:43.140523 8867 raft_consensus.cc:504] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } }
I20260925 13:07:43.140651 8867 leader_election.cc:288] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [CANDIDATE]: Term 1 election: Requested vote from peers b540febd45d340fd86f9f8cf6f1b57db (127.0.206.67:33689), 0f330d8bdff34b24804edef4ea2f5121 (127.0.206.66:42371)
I20260925 13:07:43.140829 8646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "def14d7fd3bb46b690eca4bb54a1da52" candidate_uuid: "2f14f61c0a12443eb448bbcb217b5116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f330d8bdff34b24804edef4ea2f5121"
I20260925 13:07:43.140856 8786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "def14d7fd3bb46b690eca4bb54a1da52" candidate_uuid: "2f14f61c0a12443eb448bbcb217b5116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b540febd45d340fd86f9f8cf6f1b57db"
I20260925 13:07:43.140910 8646 raft_consensus.cc:3037] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:43.140946 8786 raft_consensus.cc:3037] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:43.141625 8646 raft_consensus.cc:2454] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f14f61c0a12443eb448bbcb217b5116 in term 1.
I20260925 13:07:43.141628 8786 raft_consensus.cc:2454] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f14f61c0a12443eb448bbcb217b5116 in term 1.
I20260925 13:07:43.141793 8440 leader_election.cc:302] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f330d8bdff34b24804edef4ea2f5121, 2f14f61c0a12443eb448bbcb217b5116; no voters:
I20260925 13:07:43.141884 8867 raft_consensus.cc:2781] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:43.142074 8867 raft_consensus.cc:686] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [term 1 LEADER]: Becoming Leader. State: Replica: 2f14f61c0a12443eb448bbcb217b5116, State: Running, Role: LEADER
I20260925 13:07:43.142168 8867 consensus_queue.cc:237] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } }
I20260925 13:07:43.143055 8348 catalog_manager.cc:5495] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f14f61c0a12443eb448bbcb217b5116 (127.0.206.65). New cstate: current_term: 1 leader_uuid: "2f14f61c0a12443eb448bbcb217b5116" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } health_report { overall_health: UNKNOWN } } }
W20260925 13:07:43.157925 8553 tablet.cc:2369] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:07:43.182742 8857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.180304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43228 (local address 127.0.206.65:45125)
0925 13:07:43.180371 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:43.180373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:43.180383 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:07:43.180443 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:43.180444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:43.180463 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:43.180537 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:43.180697 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.181163 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.181945 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.182135 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.182226 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.182402 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:43.182496 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:43.182579 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:43.182619 (+ 40us) server_negotiation.cc:299] Negotiation successful
0925 13:07:43.182637 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 13:07:43.184620 8646 raft_consensus.cc:1261] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121 [term 1 FOLLOWER]: Refusing update from remote peer 2f14f61c0a12443eb448bbcb217b5116: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:43.184623 8786 raft_consensus.cc:1261] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db [term 1 FOLLOWER]: Refusing update from remote peer 2f14f61c0a12443eb448bbcb217b5116: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:43.184928 8867 consensus_queue.cc:1035] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:43.185077 8870 consensus_queue.cc:1035] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:43.188031 8874 mvcc.cc:205] Tried to move back new op lower bound from 7333239452401418240 to 7333239452231262208. Current Snapshot: MvccSnapshot[applied={T|T < 7333239452401418240 or (T in {7333239452401418240})}]
I20260925 13:07:43.189800 8348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:43.189898 8348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:43.190580 8348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3: Sending DeleteTablet for 3 replicas of tablet def14d7fd3bb46b690eca4bb54a1da52
I20260925 13:07:43.190927 8486 tablet_service.cc:1514] Processing DeleteTablet for tablet def14d7fd3bb46b690eca4bb54a1da52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:43 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:43212
I20260925 13:07:43.190999 8766 tablet_service.cc:1514] Processing DeleteTablet for tablet def14d7fd3bb46b690eca4bb54a1da52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:43 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:49924
I20260925 13:07:43.191087 8626 tablet_service.cc:1514] Processing DeleteTablet for tablet def14d7fd3bb46b690eca4bb54a1da52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:43 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:34746
I20260925 13:07:43.191290 8885 tablet_replica.cc:330] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121: stopping tablet replica
I20260925 13:07:43.191390 8885 raft_consensus.cc:2229] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:43.191354 8348 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:07:43.191494 8348 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:07:43.191500 8885 raft_consensus.cc:2258] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:43.191984 8885 ts_tablet_manager.cc:1905] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:43.191847 8884 tablet_replica.cc:330] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db: stopping tablet replica
I20260925 13:07:43.192643 8486 tablet_service.cc:1467] Processing CreateTablet for tablet b2cd99ffbfa24e4ca173d1520e1461fb (DEFAULT_TABLE table=test-table [id=d6d62800e0604b9194e58b4018e7d297]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:43.192668 8626 tablet_service.cc:1467] Processing CreateTablet for tablet b2cd99ffbfa24e4ca173d1520e1461fb (DEFAULT_TABLE table=test-table [id=d6d62800e0604b9194e58b4018e7d297]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:43.192710 8766 tablet_service.cc:1467] Processing CreateTablet for tablet b2cd99ffbfa24e4ca173d1520e1461fb (DEFAULT_TABLE table=test-table [id=d6d62800e0604b9194e58b4018e7d297]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:43.192752 8486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2cd99ffbfa24e4ca173d1520e1461fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:43.192762 8626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2cd99ffbfa24e4ca173d1520e1461fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:43.192829 8766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2cd99ffbfa24e4ca173d1520e1461fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:43.194031 8860 tablet_bootstrap.cc:492] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121: Bootstrap starting.
I20260925 13:07:43.194332 8861 tablet_bootstrap.cc:492] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db: Bootstrap starting.
I20260925 13:07:43.194461 8860 tablet_bootstrap.cc:654] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:43.192474 8884 raft_consensus.cc:2229] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:43.194787 8862 tablet_bootstrap.cc:492] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116: Bootstrap starting.
I20260925 13:07:43.194784 8861 tablet_bootstrap.cc:654] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:43.194813 8884 raft_consensus.cc:2258] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:43.195168 8862 tablet_bootstrap.cc:654] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:43.195238 8884 ts_tablet_manager.cc:1905] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:43.195827 8862 tablet_bootstrap.cc:492] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116: No bootstrap required, opened a new log
I20260925 13:07:43.195871 8862 ts_tablet_manager.cc:1397] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 13:07:43.196010 8862 raft_consensus.cc:348] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.196092 8862 raft_consensus.cc:374] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:43.196131 8862 raft_consensus.cc:729] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f14f61c0a12443eb448bbcb217b5116, State: Initialized, Role: FOLLOWER
I20260925 13:07:43.196182 8862 consensus_queue.cc:260] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.196286 8862 ts_tablet_manager.cc:1428] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:43.196616 8883 tablet_replica.cc:330] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116: stopping tablet replica
I20260925 13:07:43.196703 8883 raft_consensus.cc:2229] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:07:43.196791 8883 raft_consensus.cc:2258] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:43.197022 8885 ts_tablet_manager.cc:1918] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:07:43.197098 8885 log.cc:1196] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/def14d7fd3bb46b690eca4bb54a1da52
I20260925 13:07:43.197242 8884 ts_tablet_manager.cc:1918] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:07:43.197325 8884 log.cc:1196] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/def14d7fd3bb46b690eca4bb54a1da52
I20260925 13:07:43.197412 8885 ts_tablet_manager.cc:1939] T def14d7fd3bb46b690eca4bb54a1da52 P 0f330d8bdff34b24804edef4ea2f5121: Deleting consensus metadata
I20260925 13:07:43.197579 8884 ts_tablet_manager.cc:1939] T def14d7fd3bb46b690eca4bb54a1da52 P b540febd45d340fd86f9f8cf6f1b57db: Deleting consensus metadata
I20260925 13:07:43.197865 8336 catalog_manager.cc:4841] TS 0f330d8bdff34b24804edef4ea2f5121 (127.0.206.66:42371): tablet def14d7fd3bb46b690eca4bb54a1da52 (table test-table [id=6731887352104e85a2b3bfae4bdd4293]) successfully deleted
I20260925 13:07:43.197901 8860 tablet_bootstrap.cc:492] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121: No bootstrap required, opened a new log
I20260925 13:07:43.197937 8860 ts_tablet_manager.cc:1397] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260925 13:07:43.197952 8336 catalog_manager.cc:4841] TS b540febd45d340fd86f9f8cf6f1b57db (127.0.206.67:33689): tablet def14d7fd3bb46b690eca4bb54a1da52 (table test-table [id=6731887352104e85a2b3bfae4bdd4293]) successfully deleted
I20260925 13:07:43.198071 8860 raft_consensus.cc:348] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.198145 8860 raft_consensus.cc:374] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:43.198169 8860 raft_consensus.cc:729] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f330d8bdff34b24804edef4ea2f5121, State: Initialized, Role: FOLLOWER
I20260925 13:07:43.198212 8860 consensus_queue.cc:260] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.198256 8883 ts_tablet_manager.cc:1905] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:43.198307 8860 ts_tablet_manager.cc:1428] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 13:07:43.198807 8861 tablet_bootstrap.cc:492] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db: No bootstrap required, opened a new log
I20260925 13:07:43.198854 8861 ts_tablet_manager.cc:1397] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260925 13:07:43.199000 8861 raft_consensus.cc:348] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.199080 8861 raft_consensus.cc:374] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:43.199107 8861 raft_consensus.cc:729] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b540febd45d340fd86f9f8cf6f1b57db, State: Initialized, Role: FOLLOWER
I20260925 13:07:43.199146 8861 consensus_queue.cc:260] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.199218 8861 ts_tablet_manager.cc:1428] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:43.200083 8883 ts_tablet_manager.cc:1918] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:07:43.200152 8883 log.cc:1196] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/wal/wals/def14d7fd3bb46b690eca4bb54a1da52
I20260925 13:07:43.200395 8883 ts_tablet_manager.cc:1939] T def14d7fd3bb46b690eca4bb54a1da52 P 2f14f61c0a12443eb448bbcb217b5116: Deleting consensus metadata
I20260925 13:07:43.200739 8335 catalog_manager.cc:4841] TS 2f14f61c0a12443eb448bbcb217b5116 (127.0.206.65:45125): tablet def14d7fd3bb46b690eca4bb54a1da52 (table test-table [id=6731887352104e85a2b3bfae4bdd4293]) successfully deleted
W20260925 13:07:43.205003 8833 tablet.cc:2369] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:07:43.228125 8870 raft_consensus.cc:482] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:43.228196 8870 raft_consensus.cc:504] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.228364 8870 leader_election.cc:288] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f330d8bdff34b24804edef4ea2f5121 (127.0.206.66:42371), b540febd45d340fd86f9f8cf6f1b57db (127.0.206.67:33689)
I20260925 13:07:43.228518 8646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2cd99ffbfa24e4ca173d1520e1461fb" candidate_uuid: "2f14f61c0a12443eb448bbcb217b5116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f330d8bdff34b24804edef4ea2f5121" is_pre_election: true
I20260925 13:07:43.228542 8786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2cd99ffbfa24e4ca173d1520e1461fb" candidate_uuid: "2f14f61c0a12443eb448bbcb217b5116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b540febd45d340fd86f9f8cf6f1b57db" is_pre_election: true
I20260925 13:07:43.228642 8786 raft_consensus.cc:2454] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f14f61c0a12443eb448bbcb217b5116 in term 0.
I20260925 13:07:43.228663 8646 raft_consensus.cc:2454] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f14f61c0a12443eb448bbcb217b5116 in term 0.
I20260925 13:07:43.228777 8440 leader_election.cc:302] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f14f61c0a12443eb448bbcb217b5116, b540febd45d340fd86f9f8cf6f1b57db; no voters:
I20260925 13:07:43.228886 8870 raft_consensus.cc:2781] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:07:43.228921 8870 raft_consensus.cc:482] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:07:43.228948 8870 raft_consensus.cc:3037] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:43.229729 8870 raft_consensus.cc:504] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.229842 8870 leader_election.cc:288] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [CANDIDATE]: Term 1 election: Requested vote from peers 0f330d8bdff34b24804edef4ea2f5121 (127.0.206.66:42371), b540febd45d340fd86f9f8cf6f1b57db (127.0.206.67:33689)
I20260925 13:07:43.230012 8646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2cd99ffbfa24e4ca173d1520e1461fb" candidate_uuid: "2f14f61c0a12443eb448bbcb217b5116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f330d8bdff34b24804edef4ea2f5121"
I20260925 13:07:43.230101 8646 raft_consensus.cc:3037] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:43.230072 8786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2cd99ffbfa24e4ca173d1520e1461fb" candidate_uuid: "2f14f61c0a12443eb448bbcb217b5116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b540febd45d340fd86f9f8cf6f1b57db"
I20260925 13:07:43.230139 8786 raft_consensus.cc:3037] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:43.230809 8646 raft_consensus.cc:2454] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f14f61c0a12443eb448bbcb217b5116 in term 1.
I20260925 13:07:43.230880 8786 raft_consensus.cc:2454] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f14f61c0a12443eb448bbcb217b5116 in term 1.
I20260925 13:07:43.230993 8440 leader_election.cc:302] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f330d8bdff34b24804edef4ea2f5121, 2f14f61c0a12443eb448bbcb217b5116; no voters:
I20260925 13:07:43.231144 8870 raft_consensus.cc:2781] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:43.231215 8870 raft_consensus.cc:686] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [term 1 LEADER]: Becoming Leader. State: Replica: 2f14f61c0a12443eb448bbcb217b5116, State: Running, Role: LEADER
I20260925 13:07:43.231261 8870 consensus_queue.cc:237] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.231796 8348 catalog_manager.cc:5495] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f14f61c0a12443eb448bbcb217b5116 (127.0.206.65). New cstate: current_term: 1 leader_uuid: "2f14f61c0a12443eb448bbcb217b5116" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } health_report { overall_health: UNKNOWN } } }
Sep 25 13:07:43 dist-test-slave-chkf krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341662, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.126@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260925 13:07:43.262744 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.255394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44456 (local address 127.0.206.126:35803)
0925 13:07:43.255439 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:43.255441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:43.255534 (+ 93us) server_negotiation.cc:407] Connection header received
0925 13:07:43.255597 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:43.255598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:43.255607 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:43.255659 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:43.256190 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.256648 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.257650 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.257825 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.257892 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.260109 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:43.260110 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:43.260110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:43.260116 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:43.261552 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:43.261948 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:43.261949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:43.261950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:43.262003 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:43.262203 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:43.262204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:43.262204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:43.262304 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:43.262379 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:43.262534 (+ 155us) server_negotiation.cc:299] Negotiation successful
0925 13:07:43.262618 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 13:07:43.262846 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.255292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:35803 (local address 127.0.0.1:44456)
0925 13:07:43.255473 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:43.255486 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:43.255544 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:43.255682 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:43.255683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:43.256049 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:43.256101 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.256103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.256735 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.256736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.257564 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.257567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.257911 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.257913 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.257972 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.258491 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:43.258497 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:43.260014 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:43.261651 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:43.261658 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:43.261660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:43.261857 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:43.262076 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:43.262077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:43.262077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:43.262138 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:43.262372 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:43.262378 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:43.262421 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 13:07:43.262512 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 13:07:43.262689 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":81,"thread_start_us":102,"threads_started":1}
I20260925 13:07:43.264559 8348 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:44456:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 13:07:43.264719 8348 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:07:43.266040 8486 tablet_service.cc:1467] Processing CreateTablet for tablet 5015d3c6e750474d98cb372718208a68 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a42321f492eb46dda8eedadd777cfea5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 13:07:43.266080 8626 tablet_service.cc:1467] Processing CreateTablet for tablet 5015d3c6e750474d98cb372718208a68 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a42321f492eb46dda8eedadd777cfea5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 13:07:43.266188 8486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5015d3c6e750474d98cb372718208a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:43.266312 8626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5015d3c6e750474d98cb372718208a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:43.266846 8766 tablet_service.cc:1467] Processing CreateTablet for tablet 5015d3c6e750474d98cb372718208a68 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a42321f492eb46dda8eedadd777cfea5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 13:07:43.266979 8766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5015d3c6e750474d98cb372718208a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:43.267700 8862 tablet_bootstrap.cc:492] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116: Bootstrap starting.
I20260925 13:07:43.267756 8860 tablet_bootstrap.cc:492] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121: Bootstrap starting.
I20260925 13:07:43.268148 8860 tablet_bootstrap.cc:654] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:43.268149 8862 tablet_bootstrap.cc:654] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:43.268486 8861 tablet_bootstrap.cc:492] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db: Bootstrap starting.
I20260925 13:07:43.268798 8861 tablet_bootstrap.cc:654] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:43.269089 8860 tablet_bootstrap.cc:492] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121: No bootstrap required, opened a new log
I20260925 13:07:43.269142 8860 ts_tablet_manager.cc:1397] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 13:07:43.269308 8860 raft_consensus.cc:348] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.269402 8860 raft_consensus.cc:374] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:43.269428 8860 raft_consensus.cc:729] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f330d8bdff34b24804edef4ea2f5121, State: Initialized, Role: FOLLOWER
I20260925 13:07:43.269480 8860 consensus_queue.cc:260] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.269536 8861 tablet_bootstrap.cc:492] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db: No bootstrap required, opened a new log
I20260925 13:07:43.269573 8861 ts_tablet_manager.cc:1397] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 13:07:43.269574 8860 ts_tablet_manager.cc:1428] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:43.269631 8866 tablet_replica.cc:439] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } }
I20260925 13:07:43.269738 8866 tablet_replica.cc:442] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:43.269734 8861 raft_consensus.cc:348] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.269783 8861 raft_consensus.cc:374] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:43.269796 8861 raft_consensus.cc:729] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b540febd45d340fd86f9f8cf6f1b57db, State: Initialized, Role: FOLLOWER
I20260925 13:07:43.269826 8861 consensus_queue.cc:260] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.269918 8861 ts_tablet_manager.cc:1428] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:43.269972 8868 tablet_replica.cc:439] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } }
I20260925 13:07:43.270047 8868 tablet_replica.cc:442] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:43.270848 8862 tablet_bootstrap.cc:492] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116: No bootstrap required, opened a new log
I20260925 13:07:43.270895 8862 ts_tablet_manager.cc:1397] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260925 13:07:43.271029 8862 raft_consensus.cc:348] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.271104 8862 raft_consensus.cc:374] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:43.271150 8862 raft_consensus.cc:729] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f14f61c0a12443eb448bbcb217b5116, State: Initialized, Role: FOLLOWER
I20260925 13:07:43.271195 8862 consensus_queue.cc:260] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.271283 8862 ts_tablet_manager.cc:1428] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:43.271301 8870 tablet_replica.cc:439] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } }
I20260925 13:07:43.271361 8870 tablet_replica.cc:442] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:43.361012 8866 raft_consensus.cc:482] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:43.361107 8866 raft_consensus.cc:504] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.361382 8866 leader_election.cc:288] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f14f61c0a12443eb448bbcb217b5116 (127.0.206.65:45125), b540febd45d340fd86f9f8cf6f1b57db (127.0.206.67:33689)
I20260925 13:07:43.365252 8857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.361620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:34913 (local address 127.0.206.65:45125)
0925 13:07:43.361671 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:43.361674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:43.361753 (+ 79us) server_negotiation.cc:407] Connection header received
0925 13:07:43.361819 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:43.361819 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:43.361831 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:43.361889 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:43.362093 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.362873 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.364357 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.364892 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.364963 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.364970 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:43.365150 (+ 180us) server_negotiation.cc:299] Negotiation successful
0925 13:07:43.365183 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 13:07:43.365252 8856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.361627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:58595 (local address 127.0.206.67:33689)
0925 13:07:43.361670 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:43.361672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:43.361753 (+ 81us) server_negotiation.cc:407] Connection header received
0925 13:07:43.361806 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:43.361807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:43.361817 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:43.361889 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:43.362094 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.362859 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.364358 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.364887 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.364963 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.364970 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:43.365150 (+ 180us) server_negotiation.cc:299] Negotiation successful
0925 13:07:43.365185 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 13:07:43.365320 8899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.361552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:45125 (local address 127.0.206.66:34913)
0925 13:07:43.361670 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:43.361682 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:43.361746 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:43.361912 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:43.361913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:43.361915 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:43.362012 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.362015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.362958 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.362958 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.364262 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.364265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.364981 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.364982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.365062 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.365067 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 13:07:43.365125 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 13:07:43.365175 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":66,"thread_start_us":58,"threads_started":1}
I20260925 13:07:43.365320 8900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.361552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:33689 (local address 127.0.206.66:58595)
0925 13:07:43.361670 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:43.361682 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:43.361748 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:43.361912 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:43.361913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:43.361916 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:43.362012 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.362014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.362957 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.362958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.364262 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.364265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.364983 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.364983 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.365062 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.365068 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 13:07:43.365125 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 13:07:43.365175 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":107,"thread_start_us":54,"threads_started":1}
I20260925 13:07:43.365654 8506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5015d3c6e750474d98cb372718208a68" candidate_uuid: "0f330d8bdff34b24804edef4ea2f5121" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f14f61c0a12443eb448bbcb217b5116" is_pre_election: true
I20260925 13:07:43.365783 8506 raft_consensus.cc:2454] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f330d8bdff34b24804edef4ea2f5121 in term 0.
I20260925 13:07:43.365916 8786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5015d3c6e750474d98cb372718208a68" candidate_uuid: "0f330d8bdff34b24804edef4ea2f5121" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b540febd45d340fd86f9f8cf6f1b57db" is_pre_election: true
I20260925 13:07:43.365996 8786 raft_consensus.cc:2454] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f330d8bdff34b24804edef4ea2f5121 in term 0.
I20260925 13:07:43.365993 8579 leader_election.cc:302] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f330d8bdff34b24804edef4ea2f5121, 2f14f61c0a12443eb448bbcb217b5116; no voters:
I20260925 13:07:43.366113 8866 raft_consensus.cc:2781] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:07:43.366164 8866 raft_consensus.cc:482] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:07:43.366189 8866 raft_consensus.cc:3037] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:43.366899 8866 raft_consensus.cc:504] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.367049 8866 leader_election.cc:288] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [CANDIDATE]: Term 1 election: Requested vote from peers 2f14f61c0a12443eb448bbcb217b5116 (127.0.206.65:45125), b540febd45d340fd86f9f8cf6f1b57db (127.0.206.67:33689)
I20260925 13:07:43.367183 8506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5015d3c6e750474d98cb372718208a68" candidate_uuid: "0f330d8bdff34b24804edef4ea2f5121" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f14f61c0a12443eb448bbcb217b5116"
I20260925 13:07:43.367256 8506 raft_consensus.cc:3037] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:43.367268 8786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5015d3c6e750474d98cb372718208a68" candidate_uuid: "0f330d8bdff34b24804edef4ea2f5121" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b540febd45d340fd86f9f8cf6f1b57db"
I20260925 13:07:43.367359 8786 raft_consensus.cc:3037] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:43.367892 8506 raft_consensus.cc:2454] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f330d8bdff34b24804edef4ea2f5121 in term 1.
I20260925 13:07:43.367961 8786 raft_consensus.cc:2454] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f330d8bdff34b24804edef4ea2f5121 in term 1.
I20260925 13:07:43.368096 8579 leader_election.cc:302] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f330d8bdff34b24804edef4ea2f5121, 2f14f61c0a12443eb448bbcb217b5116; no voters:
I20260925 13:07:43.368191 8866 raft_consensus.cc:2781] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:43.368346 8866 raft_consensus.cc:686] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [term 1 LEADER]: Becoming Leader. State: Replica: 0f330d8bdff34b24804edef4ea2f5121, State: Running, Role: LEADER
I20260925 13:07:43.368448 8866 consensus_queue.cc:237] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.368741 8901 tablet_replica.cc:439] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121: TxnStatusTablet state changed. Reason: New leader 0f330d8bdff34b24804edef4ea2f5121. Latest consensus state: current_term: 1 leader_uuid: "0f330d8bdff34b24804edef4ea2f5121" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } }
I20260925 13:07:43.368847 8901 tablet_replica.cc:442] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121: This TxnStatusTablet replica's current role is: LEADER
I20260925 13:07:43.369019 8903 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 13:07:43.369132 8903 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 13:07:43.369078 8349 catalog_manager.cc:5495] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f330d8bdff34b24804edef4ea2f5121 (127.0.206.66). New cstate: current_term: 1 leader_uuid: "0f330d8bdff34b24804edef4ea2f5121" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } health_report { overall_health: UNKNOWN } } }
I20260925 13:07:43.386823 8859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.384457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.0.206.66:42371)
0925 13:07:43.384499 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:43.384500 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:43.384509 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:43.384572 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:43.384573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:43.384589 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:43.384639 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:43.384765 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.385230 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.385999 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.386201 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.386302 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.386486 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:43.386580 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:43.386662 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:43.386735 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 13:07:43.386758 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 13:07:43.386862 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.384370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:42371 (local address 127.0.0.1:34762)
0925 13:07:43.384437 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:43.384449 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:43.384492 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:43.384652 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:43.384653 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:43.384654 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 13:07:43.384698 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.384700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.385312 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.385313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.385913 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.385915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.386289 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.386289 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.386361 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.386375 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 13:07:43.386650 (+ 275us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 13:07:43.386659 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 13:07:43.386745 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 13:07:43.386786 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 13:07:43.387943 8506 raft_consensus.cc:1261] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116 [term 1 FOLLOWER]: Refusing update from remote peer 0f330d8bdff34b24804edef4ea2f5121: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:43.387959 8786 raft_consensus.cc:1261] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db [term 1 FOLLOWER]: Refusing update from remote peer 0f330d8bdff34b24804edef4ea2f5121: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:43.388191 8866 consensus_queue.cc:1035] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:43.388296 8901 consensus_queue.cc:1035] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:43.388834 8868 tablet_replica.cc:439] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db: TxnStatusTablet state changed. Reason: New leader 0f330d8bdff34b24804edef4ea2f5121. Latest consensus state: current_term: 1 leader_uuid: "0f330d8bdff34b24804edef4ea2f5121" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } }
I20260925 13:07:43.388917 8868 tablet_replica.cc:442] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:43.389186 8870 tablet_replica.cc:439] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116: TxnStatusTablet state changed. Reason: New leader 0f330d8bdff34b24804edef4ea2f5121. Latest consensus state: current_term: 1 leader_uuid: "0f330d8bdff34b24804edef4ea2f5121" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } }
I20260925 13:07:43.389283 8870 tablet_replica.cc:442] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:43.389891 8875 mvcc.cc:205] Tried to move back new op lower bound from 7333239453234917376 to 7333239453157658624. Current Snapshot: MvccSnapshot[applied={T|T < 7333239453234917376}]
I20260925 13:07:43.389992 8868 tablet_replica.cc:439] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f330d8bdff34b24804edef4ea2f5121" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } }
I20260925 13:07:43.390060 8868 tablet_replica.cc:442] T 5015d3c6e750474d98cb372718208a68 P b540febd45d340fd86f9f8cf6f1b57db: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:43.390146 8866 tablet_replica.cc:439] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f330d8bdff34b24804edef4ea2f5121" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } }
I20260925 13:07:43.390216 8866 tablet_replica.cc:442] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121: This TxnStatusTablet replica's current role is: LEADER
I20260925 13:07:43.390529 8873 mvcc.cc:205] Tried to move back new op lower bound from 7333239453234917376 to 7333239453157658624. Current Snapshot: MvccSnapshot[applied={T|T < 7333239453234917376 or (T in {7333239453234917376})}]
I20260925 13:07:43.390617 8870 tablet_replica.cc:439] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f330d8bdff34b24804edef4ea2f5121" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } }
I20260925 13:07:43.390676 8870 tablet_replica.cc:442] T 5015d3c6e750474d98cb372718208a68 P 2f14f61c0a12443eb448bbcb217b5116: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 13:07:43.391269 8908 tablet_replica.cc:439] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f330d8bdff34b24804edef4ea2f5121" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } } }
I20260925 13:07:43.391464 8908 tablet_replica.cc:442] T 5015d3c6e750474d98cb372718208a68 P 0f330d8bdff34b24804edef4ea2f5121: This TxnStatusTablet replica's current role is: LEADER
I20260925 13:07:43.395469 8859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.393017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:56259 (local address 127.0.206.66:42371)
0925 13:07:43.393056 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:43.393058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:43.393127 (+ 69us) server_negotiation.cc:407] Connection header received
0925 13:07:43.393197 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:43.393198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:43.393206 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:43.393244 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:43.393364 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.393769 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.394887 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.395056 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.395103 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.395279 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:43.395301 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:43.395338 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:43.395414 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 13:07:43.395427 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 13:07:43.395538 8911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.392930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:42371 (local address 127.0.206.65:56259)
0925 13:07:43.393072 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:43.393083 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:43.393143 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:43.393255 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:43.393256 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:43.393258 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 13:07:43.393307 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.393308 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.393836 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.393837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.394789 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.394791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.395124 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.395125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.395189 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.395203 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 13:07:43.395359 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 13:07:43.395361 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:07:43.395413 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 13:07:43.395457 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260925 13:07:43.397526 8786 raft_consensus.cc:1261] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db [term 1 FOLLOWER]: Refusing update from remote peer 2f14f61c0a12443eb448bbcb217b5116: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:43.397615 8646 raft_consensus.cc:1261] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121 [term 1 FOLLOWER]: Refusing update from remote peer 2f14f61c0a12443eb448bbcb217b5116: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:43.397701 8867 consensus_queue.cc:1035] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:43.397800 8870 consensus_queue.cc:1035] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:43.405174 8915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.402496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:45125 (local address 127.0.206.66:55953)
0925 13:07:43.402662 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:43.402673 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:43.402749 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:43.402915 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:43.402916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:43.402917 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 13:07:43.402966 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.402967 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.403647 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.403648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.404303 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.404305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.404782 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.404783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.404842 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.404855 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 13:07:43.405045 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 13:07:43.405047 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:07:43.405081 (+ 34us) client_negotiation.cc:240] Negotiation successful
0925 13:07:43.405111 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260925 13:07:43.405170 8857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.402679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:55953 (local address 127.0.206.65:45125)
0925 13:07:43.402736 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:43.402738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:43.402747 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:43.402808 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:43.402809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:43.402818 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:43.402884 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:43.403047 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.403562 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.404394 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.404668 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.404727 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.404946 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:43.404982 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:43.405037 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:43.405084 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 13:07:43.405104 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 13:07:43.414551 8349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:43.414649 8349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:43.415287 8349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3: Sending DeleteTablet for 3 replicas of tablet b2cd99ffbfa24e4ca173d1520e1461fb
I20260925 13:07:43.415531 8486 tablet_service.cc:1514] Processing DeleteTablet for tablet b2cd99ffbfa24e4ca173d1520e1461fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:43 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:43212
I20260925 13:07:43.415580 8766 tablet_service.cc:1514] Processing DeleteTablet for tablet b2cd99ffbfa24e4ca173d1520e1461fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:43 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:49924
I20260925 13:07:43.415624 8883 tablet_replica.cc:330] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116: stopping tablet replica
I20260925 13:07:43.415668 8883 raft_consensus.cc:2229] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:07:43.415676 8884 tablet_replica.cc:330] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db: stopping tablet replica
I20260925 13:07:43.415724 8884 raft_consensus.cc:2229] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:43.415727 8883 raft_consensus.cc:2258] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:43.415766 8884 raft_consensus.cc:2258] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:43.415863 8626 tablet_service.cc:1514] Processing DeleteTablet for tablet b2cd99ffbfa24e4ca173d1520e1461fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:43 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:34746
I20260925 13:07:43.415946 8885 tablet_replica.cc:330] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121: stopping tablet replica
I20260925 13:07:43.415987 8885 raft_consensus.cc:2229] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:43.416033 8885 raft_consensus.cc:2258] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:43.416078 8884 ts_tablet_manager.cc:1905] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:43.416087 8883 ts_tablet_manager.cc:1905] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:43.416339 8885 ts_tablet_manager.cc:1905] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:43.417675 8884 ts_tablet_manager.cc:1918] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 13:07:43.417675 8883 ts_tablet_manager.cc:1918] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 13:07:43.417732 8884 log.cc:1196] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/b2cd99ffbfa24e4ca173d1520e1461fb
I20260925 13:07:43.417732 8883 log.cc:1196] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/wal/wals/b2cd99ffbfa24e4ca173d1520e1461fb
I20260925 13:07:43.417958 8884 ts_tablet_manager.cc:1939] T b2cd99ffbfa24e4ca173d1520e1461fb P b540febd45d340fd86f9f8cf6f1b57db: Deleting consensus metadata
I20260925 13:07:43.417995 8883 ts_tablet_manager.cc:1939] T b2cd99ffbfa24e4ca173d1520e1461fb P 2f14f61c0a12443eb448bbcb217b5116: Deleting consensus metadata
I20260925 13:07:43.418303 8335 catalog_manager.cc:4841] TS 2f14f61c0a12443eb448bbcb217b5116 (127.0.206.65:45125): tablet b2cd99ffbfa24e4ca173d1520e1461fb (table test-table [id=d6d62800e0604b9194e58b4018e7d297]) successfully deleted
I20260925 13:07:43.418375 8885 ts_tablet_manager.cc:1918] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 13:07:43.418419 8885 log.cc:1196] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/b2cd99ffbfa24e4ca173d1520e1461fb
I20260925 13:07:43.418648 8885 ts_tablet_manager.cc:1939] T b2cd99ffbfa24e4ca173d1520e1461fb P 0f330d8bdff34b24804edef4ea2f5121: Deleting consensus metadata
I20260925 13:07:43.419013 8336 catalog_manager.cc:4841] TS 0f330d8bdff34b24804edef4ea2f5121 (127.0.206.66:42371): tablet b2cd99ffbfa24e4ca173d1520e1461fb (table test-table [id=d6d62800e0604b9194e58b4018e7d297]) successfully deleted
I20260925 13:07:43.419095 8336 catalog_manager.cc:4841] TS b540febd45d340fd86f9f8cf6f1b57db (127.0.206.67:33689): tablet b2cd99ffbfa24e4ca173d1520e1461fb (table test-table [id=d6d62800e0604b9194e58b4018e7d297]) successfully deleted
Sep 25 13:07:43 dist-test-slave-chkf krb5kdc[8304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 13:07:43 dist-test-slave-chkf krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:43 dist-test-slave-chkf krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
W20260925 13:07:43.446355 8399 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:43.436080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44468 (local address 127.0.206.126:35803)
0925 13:07:43.436149 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:43.436151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:43.438971 (+ 2820us) server_negotiation.cc:407] Connection header received
0925 13:07:43.439105 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:43.439106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:43.439138 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:43.439169 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:43.440262 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.441142 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.442248 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.442590 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.442678 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.446273 (+ 3595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44468: BlockingRecv error: recv got EOF from 127.0.0.1:44468 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 25 13:07:43 dist-test-slave-chkf krb5kdc[8304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260925 13:07:43.458006 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.447389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44472 (local address 127.0.206.126:35803)
0925 13:07:43.447424 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:43.447426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:43.450884 (+ 3458us) server_negotiation.cc:407] Connection header received
0925 13:07:43.450947 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:43.450948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:43.450958 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:43.451013 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:43.451515 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.451990 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.452712 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.452920 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.453009 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.455362 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:43.455363 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:43.455364 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:43.455370 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:43.456874 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:43.457241 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:43.457243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:43.457243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:43.457306 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:43.457508 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:43.457509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:43.457510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:43.457617 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:43.457726 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:43.457836 (+ 110us) server_negotiation.cc:299] Negotiation successful
0925 13:07:43.457908 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 13:07:43.459648 8349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:07:43.459789 8349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:07:43.461247 8486 tablet_service.cc:1467] Processing CreateTablet for tablet c83f5acc1d2444259676a02c9579b2f9 (DEFAULT_TABLE table=test-table [id=dfb2565b478e48a3ba9b295e9416cf2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:43.461284 8766 tablet_service.cc:1467] Processing CreateTablet for tablet c83f5acc1d2444259676a02c9579b2f9 (DEFAULT_TABLE table=test-table [id=dfb2565b478e48a3ba9b295e9416cf2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:43.461395 8486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c83f5acc1d2444259676a02c9579b2f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:43.461447 8766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c83f5acc1d2444259676a02c9579b2f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:43.462427 8861 tablet_bootstrap.cc:492] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db: Bootstrap starting.
I20260925 13:07:43.462877 8861 tablet_bootstrap.cc:654] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:43.462882 8626 tablet_service.cc:1467] Processing CreateTablet for tablet c83f5acc1d2444259676a02c9579b2f9 (DEFAULT_TABLE table=test-table [id=dfb2565b478e48a3ba9b295e9416cf2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:43.462992 8626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c83f5acc1d2444259676a02c9579b2f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:43.463757 8861 tablet_bootstrap.cc:492] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db: No bootstrap required, opened a new log
I20260925 13:07:43.463824 8861 ts_tablet_manager.cc:1397] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 13:07:43.463974 8861 raft_consensus.cc:348] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.464057 8861 raft_consensus.cc:374] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:43.464084 8861 raft_consensus.cc:729] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b540febd45d340fd86f9f8cf6f1b57db, State: Initialized, Role: FOLLOWER
I20260925 13:07:43.464123 8861 consensus_queue.cc:260] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.464216 8861 ts_tablet_manager.cc:1428] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 13:07:43.464353 8862 tablet_bootstrap.cc:492] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116: Bootstrap starting.
I20260925 13:07:43.464677 8862 tablet_bootstrap.cc:654] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:43.464937 8860 tablet_bootstrap.cc:492] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121: Bootstrap starting.
I20260925 13:07:43.465348 8860 tablet_bootstrap.cc:654] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:43.465986 8860 tablet_bootstrap.cc:492] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121: No bootstrap required, opened a new log
I20260925 13:07:43.466001 8862 tablet_bootstrap.cc:492] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116: No bootstrap required, opened a new log
I20260925 13:07:43.466040 8860 ts_tablet_manager.cc:1397] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 13:07:43.466046 8862 ts_tablet_manager.cc:1397] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:07:43.466146 8862 raft_consensus.cc:348] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.466221 8862 raft_consensus.cc:374] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:43.466244 8862 raft_consensus.cc:729] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f14f61c0a12443eb448bbcb217b5116, State: Initialized, Role: FOLLOWER
I20260925 13:07:43.466192 8860 raft_consensus.cc:348] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.466282 8860 raft_consensus.cc:374] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:43.466285 8862 consensus_queue.cc:260] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.466308 8860 raft_consensus.cc:729] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f330d8bdff34b24804edef4ea2f5121, State: Initialized, Role: FOLLOWER
I20260925 13:07:43.466364 8860 consensus_queue.cc:260] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.466382 8862 ts_tablet_manager.cc:1428] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:43.466452 8860 ts_tablet_manager.cc:1428] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 13:07:43.507455 8901 raft_consensus.cc:482] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:43.507553 8901 raft_consensus.cc:504] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.507735 8901 leader_election.cc:288] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f14f61c0a12443eb448bbcb217b5116 (127.0.206.65:45125), b540febd45d340fd86f9f8cf6f1b57db (127.0.206.67:33689)
I20260925 13:07:43.507906 8786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c83f5acc1d2444259676a02c9579b2f9" candidate_uuid: "0f330d8bdff34b24804edef4ea2f5121" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b540febd45d340fd86f9f8cf6f1b57db" is_pre_election: true
I20260925 13:07:43.507906 8506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c83f5acc1d2444259676a02c9579b2f9" candidate_uuid: "0f330d8bdff34b24804edef4ea2f5121" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f14f61c0a12443eb448bbcb217b5116" is_pre_election: true
I20260925 13:07:43.508010 8786 raft_consensus.cc:2454] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f330d8bdff34b24804edef4ea2f5121 in term 0.
I20260925 13:07:43.508011 8506 raft_consensus.cc:2454] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f330d8bdff34b24804edef4ea2f5121 in term 0.
I20260925 13:07:43.508196 8580 leader_election.cc:302] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0f330d8bdff34b24804edef4ea2f5121, 2f14f61c0a12443eb448bbcb217b5116, b540febd45d340fd86f9f8cf6f1b57db; no voters:
I20260925 13:07:43.508322 8901 raft_consensus.cc:2781] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:07:43.508360 8901 raft_consensus.cc:482] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:07:43.508404 8901 raft_consensus.cc:3037] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:43.509306 8901 raft_consensus.cc:504] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.509447 8901 leader_election.cc:288] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [CANDIDATE]: Term 1 election: Requested vote from peers 2f14f61c0a12443eb448bbcb217b5116 (127.0.206.65:45125), b540febd45d340fd86f9f8cf6f1b57db (127.0.206.67:33689)
I20260925 13:07:43.509578 8506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c83f5acc1d2444259676a02c9579b2f9" candidate_uuid: "0f330d8bdff34b24804edef4ea2f5121" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f14f61c0a12443eb448bbcb217b5116"
I20260925 13:07:43.509662 8506 raft_consensus.cc:3037] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:43.509773 8786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c83f5acc1d2444259676a02c9579b2f9" candidate_uuid: "0f330d8bdff34b24804edef4ea2f5121" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b540febd45d340fd86f9f8cf6f1b57db"
I20260925 13:07:43.509856 8786 raft_consensus.cc:3037] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:43.510361 8506 raft_consensus.cc:2454] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f330d8bdff34b24804edef4ea2f5121 in term 1.
I20260925 13:07:43.510545 8579 leader_election.cc:302] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f330d8bdff34b24804edef4ea2f5121, 2f14f61c0a12443eb448bbcb217b5116; no voters:
I20260925 13:07:43.510591 8786 raft_consensus.cc:2454] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f330d8bdff34b24804edef4ea2f5121 in term 1.
I20260925 13:07:43.510650 8901 raft_consensus.cc:2781] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:43.510718 8901 raft_consensus.cc:686] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [term 1 LEADER]: Becoming Leader. State: Replica: 0f330d8bdff34b24804edef4ea2f5121, State: Running, Role: LEADER
I20260925 13:07:43.510778 8901 consensus_queue.cc:237] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.511350 8348 catalog_manager.cc:5495] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f330d8bdff34b24804edef4ea2f5121 (127.0.206.66). New cstate: current_term: 1 leader_uuid: "0f330d8bdff34b24804edef4ea2f5121" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } health_report { overall_health: UNKNOWN } } }
I20260925 13:07:43.523276 8859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.521037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34772 (local address 127.0.206.66:42371)
0925 13:07:43.521082 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:43.521084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:43.521093 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:43.521127 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:43.521128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:43.521137 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:43.521182 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:43.521291 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.521754 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.522530 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.522795 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.522872 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.523031 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:43.523063 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:43.523114 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:43.523181 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 13:07:43.523199 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 13:07:43.524000 8506 raft_consensus.cc:1261] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116 [term 1 FOLLOWER]: Refusing update from remote peer 0f330d8bdff34b24804edef4ea2f5121: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:43.524024 8786 raft_consensus.cc:1261] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db [term 1 FOLLOWER]: Refusing update from remote peer 0f330d8bdff34b24804edef4ea2f5121: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:43.524214 8901 consensus_queue.cc:1035] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:43.524341 8866 consensus_queue.cc:1035] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:43.527341 8348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:43.527412 8348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:43.528007 8348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3: Sending DeleteTablet for 3 replicas of tablet c83f5acc1d2444259676a02c9579b2f9
I20260925 13:07:43.528219 8486 tablet_service.cc:1514] Processing DeleteTablet for tablet c83f5acc1d2444259676a02c9579b2f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:43 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:43212
I20260925 13:07:43.528272 8626 tablet_service.cc:1514] Processing DeleteTablet for tablet c83f5acc1d2444259676a02c9579b2f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:43 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:34746
I20260925 13:07:43.528306 8883 tablet_replica.cc:330] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116: stopping tablet replica
I20260925 13:07:43.528347 8883 raft_consensus.cc:2229] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:43.528396 8883 raft_consensus.cc:2258] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:43.528472 8885 tablet_replica.cc:330] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121: stopping tablet replica
I20260925 13:07:43.528527 8885 raft_consensus.cc:2229] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:07:43.528606 8885 raft_consensus.cc:2258] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:43.528656 8766 tablet_service.cc:1514] Processing DeleteTablet for tablet c83f5acc1d2444259676a02c9579b2f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:43 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:49924
I20260925 13:07:43.528733 8884 tablet_replica.cc:330] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db: stopping tablet replica
I20260925 13:07:43.528661 8348 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 13:07:43.528775 8884 raft_consensus.cc:2229] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 13:07:43.528797 8348 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:07:43.528815 8884 raft_consensus.cc:2258] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:43.529006 8885 ts_tablet_manager.cc:1905] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:43.529366 8884 ts_tablet_manager.cc:1905] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:43.529904 8626 tablet_service.cc:1467] Processing CreateTablet for tablet 34d66081d90e4bb6b56b8f5d0cd8934f (DEFAULT_TABLE table=test-table [id=b55fc3e161f74f7787f977ca967f71d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:43.530004 8626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34d66081d90e4bb6b56b8f5d0cd8934f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:43.530646 8883 ts_tablet_manager.cc:1905] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:43.530766 8884 ts_tablet_manager.cc:1918] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:07:43.530813 8884 log.cc:1196] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/c83f5acc1d2444259676a02c9579b2f9
I20260925 13:07:43.531011 8884 ts_tablet_manager.cc:1939] T c83f5acc1d2444259676a02c9579b2f9 P b540febd45d340fd86f9f8cf6f1b57db: Deleting consensus metadata
I20260925 13:07:43.531136 8766 tablet_service.cc:1467] Processing CreateTablet for tablet 34d66081d90e4bb6b56b8f5d0cd8934f (DEFAULT_TABLE table=test-table [id=b55fc3e161f74f7787f977ca967f71d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:43.531204 8766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34d66081d90e4bb6b56b8f5d0cd8934f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:43.532608 8883 ts_tablet_manager.cc:1918] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:07:43.532660 8883 log.cc:1196] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-0/wal/wals/c83f5acc1d2444259676a02c9579b2f9
I20260925 13:07:43.532858 8861 tablet_bootstrap.cc:492] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db: Bootstrap starting.
I20260925 13:07:43.532879 8883 ts_tablet_manager.cc:1939] T c83f5acc1d2444259676a02c9579b2f9 P 2f14f61c0a12443eb448bbcb217b5116: Deleting consensus metadata
I20260925 13:07:43.533103 8486 tablet_service.cc:1467] Processing CreateTablet for tablet 34d66081d90e4bb6b56b8f5d0cd8934f (DEFAULT_TABLE table=test-table [id=b55fc3e161f74f7787f977ca967f71d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:07:43.533203 8486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34d66081d90e4bb6b56b8f5d0cd8934f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:43.533288 8861 tablet_bootstrap.cc:654] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:43.533948 8861 tablet_bootstrap.cc:492] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db: No bootstrap required, opened a new log
I20260925 13:07:43.534000 8861 ts_tablet_manager.cc:1397] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260925 13:07:43.534125 8861 raft_consensus.cc:348] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.534189 8861 raft_consensus.cc:374] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:43.534209 8861 raft_consensus.cc:729] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b540febd45d340fd86f9f8cf6f1b57db, State: Initialized, Role: FOLLOWER
I20260925 13:07:43.534252 8861 consensus_queue.cc:260] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.534314 8862 tablet_bootstrap.cc:492] T 34d66081d90e4bb6b56b8f5d0cd8934f P 2f14f61c0a12443eb448bbcb217b5116: Bootstrap starting.
I20260925 13:07:43.534359 8861 ts_tablet_manager.cc:1428] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:43.534626 8862 tablet_bootstrap.cc:654] T 34d66081d90e4bb6b56b8f5d0cd8934f P 2f14f61c0a12443eb448bbcb217b5116: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:43.535012 8885 ts_tablet_manager.cc:1918] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:07:43.535069 8885 log.cc:1196] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/c83f5acc1d2444259676a02c9579b2f9
I20260925 13:07:43.535280 8885 ts_tablet_manager.cc:1939] T c83f5acc1d2444259676a02c9579b2f9 P 0f330d8bdff34b24804edef4ea2f5121: Deleting consensus metadata
I20260925 13:07:43.535575 8336 catalog_manager.cc:4841] TS 0f330d8bdff34b24804edef4ea2f5121 (127.0.206.66:42371): tablet c83f5acc1d2444259676a02c9579b2f9 (table test-table [id=dfb2565b478e48a3ba9b295e9416cf2c]) successfully deleted
I20260925 13:07:43.535668 8860 tablet_bootstrap.cc:492] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121: Bootstrap starting.
I20260925 13:07:43.535848 8862 tablet_bootstrap.cc:492] T 34d66081d90e4bb6b56b8f5d0cd8934f P 2f14f61c0a12443eb448bbcb217b5116: No bootstrap required, opened a new log
I20260925 13:07:43.535902 8862 ts_tablet_manager.cc:1397] T 34d66081d90e4bb6b56b8f5d0cd8934f P 2f14f61c0a12443eb448bbcb217b5116: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:07:43.535960 8860 tablet_bootstrap.cc:654] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:43.536010 8862 raft_consensus.cc:348] T 34d66081d90e4bb6b56b8f5d0cd8934f P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.536090 8862 raft_consensus.cc:374] T 34d66081d90e4bb6b56b8f5d0cd8934f P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:43.536118 8862 raft_consensus.cc:729] T 34d66081d90e4bb6b56b8f5d0cd8934f P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f14f61c0a12443eb448bbcb217b5116, State: Initialized, Role: FOLLOWER
I20260925 13:07:43.536167 8862 consensus_queue.cc:260] T 34d66081d90e4bb6b56b8f5d0cd8934f P 2f14f61c0a12443eb448bbcb217b5116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.536264 8862 ts_tablet_manager.cc:1428] T 34d66081d90e4bb6b56b8f5d0cd8934f P 2f14f61c0a12443eb448bbcb217b5116: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:43.536360 8336 catalog_manager.cc:4841] TS b540febd45d340fd86f9f8cf6f1b57db (127.0.206.67:33689): tablet c83f5acc1d2444259676a02c9579b2f9 (table test-table [id=dfb2565b478e48a3ba9b295e9416cf2c]) successfully deleted
I20260925 13:07:43.536646 8335 catalog_manager.cc:4841] TS 2f14f61c0a12443eb448bbcb217b5116 (127.0.206.65:45125): tablet c83f5acc1d2444259676a02c9579b2f9 (table test-table [id=dfb2565b478e48a3ba9b295e9416cf2c]) successfully deleted
I20260925 13:07:43.537097 8860 tablet_bootstrap.cc:492] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121: No bootstrap required, opened a new log
I20260925 13:07:43.537137 8860 ts_tablet_manager.cc:1397] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260925 13:07:43.537243 8860 raft_consensus.cc:348] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.537307 8860 raft_consensus.cc:374] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:43.537321 8860 raft_consensus.cc:729] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f330d8bdff34b24804edef4ea2f5121, State: Initialized, Role: FOLLOWER
I20260925 13:07:43.537379 8860 consensus_queue.cc:260] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.537469 8860 ts_tablet_manager.cc:1428] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 13:07:43.835543 8868 raft_consensus.cc:482] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:07:43.835686 8868 raft_consensus.cc:504] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.835991 8868 leader_election.cc:288] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f14f61c0a12443eb448bbcb217b5116 (127.0.206.65:45125), 0f330d8bdff34b24804edef4ea2f5121 (127.0.206.66:42371)
I20260925 13:07:43.840046 8857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.836250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:52607 (local address 127.0.206.65:45125)
0925 13:07:43.836300 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:43.836302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:43.836401 (+ 99us) server_negotiation.cc:407] Connection header received
0925 13:07:43.836443 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:43.836444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:43.836456 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:43.836528 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:43.836738 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.837473 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.839131 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.839678 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.839753 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.839762 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:43.839943 (+ 181us) server_negotiation.cc:299] Negotiation successful
0925 13:07:43.839980 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 13:07:43.840046 8859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.836261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:42023 (local address 127.0.206.66:42371)
0925 13:07:43.836300 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:43.836302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:43.836401 (+ 99us) server_negotiation.cc:407] Connection header received
0925 13:07:43.836443 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:43.836444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:43.836455 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:43.836528 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:07:43.836738 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.837479 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.839131 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.839679 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.839753 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.839762 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:43.839943 (+ 181us) server_negotiation.cc:299] Negotiation successful
0925 13:07:43.839977 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 13:07:43.840098 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.836178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:45125 (local address 127.0.206.67:52607)
0925 13:07:43.836310 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:43.836322 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:43.836392 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:43.836553 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:43.836554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:43.836557 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:43.836657 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.836659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.837590 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.837591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.839039 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.839042 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.839765 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.839766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.839849 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.839853 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:07:43.839916 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 13:07:43.839970 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":59,"thread_start_us":62,"threads_started":1}
I20260925 13:07:43.840104 8940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.836178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:42371 (local address 127.0.206.67:42023)
0925 13:07:43.836311 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:43.836322 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:43.836392 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:43.836553 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:43.836554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:43.836557 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:07:43.836657 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.836660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.837590 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.837591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.839040 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.839042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.839764 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.839765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.839849 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.839855 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 13:07:43.839916 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 13:07:43.839970 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":127,"thread_start_us":62,"threads_started":1}
I20260925 13:07:43.840415 8506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34d66081d90e4bb6b56b8f5d0cd8934f" candidate_uuid: "b540febd45d340fd86f9f8cf6f1b57db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f14f61c0a12443eb448bbcb217b5116" is_pre_election: true
I20260925 13:07:43.840416 8646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34d66081d90e4bb6b56b8f5d0cd8934f" candidate_uuid: "b540febd45d340fd86f9f8cf6f1b57db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f330d8bdff34b24804edef4ea2f5121" is_pre_election: true
I20260925 13:07:43.840502 8506 raft_consensus.cc:2454] T 34d66081d90e4bb6b56b8f5d0cd8934f P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b540febd45d340fd86f9f8cf6f1b57db in term 0.
I20260925 13:07:43.840533 8646 raft_consensus.cc:2454] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b540febd45d340fd86f9f8cf6f1b57db in term 0.
I20260925 13:07:43.840667 8719 leader_election.cc:302] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f14f61c0a12443eb448bbcb217b5116, b540febd45d340fd86f9f8cf6f1b57db; no voters:
I20260925 13:07:43.840804 8868 raft_consensus.cc:2781] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:07:43.840844 8868 raft_consensus.cc:482] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:07:43.840881 8868 raft_consensus.cc:3037] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:43.841501 8868 raft_consensus.cc:504] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.841641 8868 leader_election.cc:288] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [CANDIDATE]: Term 1 election: Requested vote from peers 2f14f61c0a12443eb448bbcb217b5116 (127.0.206.65:45125), 0f330d8bdff34b24804edef4ea2f5121 (127.0.206.66:42371)
I20260925 13:07:43.841786 8506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34d66081d90e4bb6b56b8f5d0cd8934f" candidate_uuid: "b540febd45d340fd86f9f8cf6f1b57db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f14f61c0a12443eb448bbcb217b5116"
I20260925 13:07:43.841864 8506 raft_consensus.cc:3037] T 34d66081d90e4bb6b56b8f5d0cd8934f P 2f14f61c0a12443eb448bbcb217b5116 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:43.841925 8646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34d66081d90e4bb6b56b8f5d0cd8934f" candidate_uuid: "b540febd45d340fd86f9f8cf6f1b57db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f330d8bdff34b24804edef4ea2f5121"
I20260925 13:07:43.842002 8646 raft_consensus.cc:3037] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:43.842533 8506 raft_consensus.cc:2454] T 34d66081d90e4bb6b56b8f5d0cd8934f P 2f14f61c0a12443eb448bbcb217b5116 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b540febd45d340fd86f9f8cf6f1b57db in term 1.
I20260925 13:07:43.842648 8646 raft_consensus.cc:2454] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b540febd45d340fd86f9f8cf6f1b57db in term 1.
I20260925 13:07:43.842768 8719 leader_election.cc:302] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f14f61c0a12443eb448bbcb217b5116, b540febd45d340fd86f9f8cf6f1b57db; no voters:
I20260925 13:07:43.842882 8868 raft_consensus.cc:2781] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:43.843015 8868 raft_consensus.cc:686] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [term 1 LEADER]: Becoming Leader. State: Replica: b540febd45d340fd86f9f8cf6f1b57db, State: Running, Role: LEADER
I20260925 13:07:43.843112 8868 consensus_queue.cc:237] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } }
I20260925 13:07:43.843775 8348 catalog_manager.cc:5495] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db reported cstate change: term changed from 0 to 1, leader changed from <none> to b540febd45d340fd86f9f8cf6f1b57db (127.0.206.67). New cstate: current_term: 1 leader_uuid: "b540febd45d340fd86f9f8cf6f1b57db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b540febd45d340fd86f9f8cf6f1b57db" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33689 } health_report { overall_health: HEALTHY } } }
I20260925 13:07:43.887990 8943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.885643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49936 (local address 127.0.206.67:33689)
0925 13:07:43.885784 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:43.885786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:43.885794 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:43.885844 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:43.885845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:43.885879 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:43.885948 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:43.886041 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.886496 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.887353 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.887534 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.887587 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.887701 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:43.887822 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:43.887874 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:43.887917 (+ 43us) server_negotiation.cc:299] Negotiation successful
0925 13:07:43.887931 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":101,"threads_started":1}
I20260925 13:07:43.892318 8859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.890206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:40813 (local address 127.0.206.66:42371)
0925 13:07:43.890275 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:43.890276 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:43.890321 (+ 45us) server_negotiation.cc:407] Connection header received
0925 13:07:43.890380 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:43.890380 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:43.890388 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:43.890431 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:43.890548 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.891008 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.891748 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.891928 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.891978 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.892136 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:43.892159 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:43.892195 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:43.892252 (+ 57us) server_negotiation.cc:299] Negotiation successful
0925 13:07:43.892267 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 13:07:43.892360 8946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.890137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:42371 (local address 127.0.206.67:40813)
0925 13:07:43.890268 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:43.890283 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:43.890329 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:43.890441 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:43.890441 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:43.890443 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 13:07:43.890494 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.890496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.891078 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.891078 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.891669 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.891671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.892010 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.892011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.892054 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.892064 (+ 10us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 13:07:43.892214 (+ 150us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 13:07:43.892216 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:07:43.892251 (+ 35us) client_negotiation.cc:240] Negotiation successful
0925 13:07:43.892285 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
I20260925 13:07:43.894214 8646 raft_consensus.cc:1261] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121 [term 1 FOLLOWER]: Refusing update from remote peer b540febd45d340fd86f9f8cf6f1b57db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:43.894207 8506 raft_consensus.cc:1261] T 34d66081d90e4bb6b56b8f5d0cd8934f P 2f14f61c0a12443eb448bbcb217b5116 [term 1 FOLLOWER]: Refusing update from remote peer b540febd45d340fd86f9f8cf6f1b57db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:07:43.894481 8868 consensus_queue.cc:1035] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f14f61c0a12443eb448bbcb217b5116" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:43.894636 8941 consensus_queue.cc:1035] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f330d8bdff34b24804edef4ea2f5121" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:07:43.901356 8943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.899375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:39249 (local address 127.0.206.67:33689)
0925 13:07:43.899435 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:43.899437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:43.899445 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:43.899466 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:43.899467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:43.899476 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:43.899529 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:07:43.899632 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.900042 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.900678 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.900892 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.900955 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.901098 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:07:43.901126 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:07:43.901210 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:43.901272 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 13:07:43.901289 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 13:07:43.901402 8915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.899235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:33689 (local address 127.0.206.66:39249)
0925 13:07:43.899268 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:43.899277 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:43.899311 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:43.899544 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:43.899546 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:43.899547 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 13:07:43.899584 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.899584 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.900096 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.900097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.900609 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:43.900610 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:43.900966 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:43.900967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:43.901022 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:43.901029 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 13:07:43.901213 (+ 184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 13:07:43.901215 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:07:43.901268 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 13:07:43.901313 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260925 13:07:43.908022 8348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:43.908106 8348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:07:43.908651 8348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6891f5172964bd8bb9acca9879009c3: Sending DeleteTablet for 3 replicas of tablet 34d66081d90e4bb6b56b8f5d0cd8934f
I20260925 13:07:43.908931 8486 tablet_service.cc:1514] Processing DeleteTablet for tablet 34d66081d90e4bb6b56b8f5d0cd8934f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:43 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:43212
I20260925 13:07:43.909046 8883 tablet_replica.cc:330] T 34d66081d90e4bb6b56b8f5d0cd8934f P 2f14f61c0a12443eb448bbcb217b5116: stopping tablet replica
I20260925 13:07:43.909094 8883 raft_consensus.cc:2229] T 34d66081d90e4bb6b56b8f5d0cd8934f P 2f14f61c0a12443eb448bbcb217b5116 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:43.909133 8883 raft_consensus.cc:2258] T 34d66081d90e4bb6b56b8f5d0cd8934f P 2f14f61c0a12443eb448bbcb217b5116 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:43.909272 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 8421
I20260925 13:07:43.911316 8626 tablet_service.cc:1514] Processing DeleteTablet for tablet 34d66081d90e4bb6b56b8f5d0cd8934f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:43 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:34746
I20260925 13:07:43.911334 8766 tablet_service.cc:1514] Processing DeleteTablet for tablet 34d66081d90e4bb6b56b8f5d0cd8934f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:07:43 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:49924
I20260925 13:07:43.911432 8885 tablet_replica.cc:330] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121: stopping tablet replica
I20260925 13:07:43.911495 8885 raft_consensus.cc:2229] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:07:43.911535 8885 raft_consensus.cc:2258] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:43.911605 8884 tablet_replica.cc:330] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db: stopping tablet replica
I20260925 13:07:43.911659 8884 raft_consensus.cc:2229] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:07:43.911737 8884 raft_consensus.cc:2258] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:07:43.912063 8885 ts_tablet_manager.cc:1905] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:43.913853 8885 ts_tablet_manager.cc:1918] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 13:07:43.913918 8885 log.cc:1196] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/34d66081d90e4bb6b56b8f5d0cd8934f
I20260925 13:07:43.914120 8885 ts_tablet_manager.cc:1939] T 34d66081d90e4bb6b56b8f5d0cd8934f P 0f330d8bdff34b24804edef4ea2f5121: Deleting consensus metadata
I20260925 13:07:43.914505 8336 catalog_manager.cc:4841] TS 0f330d8bdff34b24804edef4ea2f5121 (127.0.206.66:42371): tablet 34d66081d90e4bb6b56b8f5d0cd8934f (table test-table [id=b55fc3e161f74f7787f977ca967f71d5]) successfully deleted
I20260925 13:07:43.914968 8884 ts_tablet_manager.cc:1905] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:07:43.916689 8884 ts_tablet_manager.cc:1918] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 13:07:43.916749 8884 log.cc:1196] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/34d66081d90e4bb6b56b8f5d0cd8934f
I20260925 13:07:43.916958 8884 ts_tablet_manager.cc:1939] T 34d66081d90e4bb6b56b8f5d0cd8934f P b540febd45d340fd86f9f8cf6f1b57db: Deleting consensus metadata
W20260925 13:07:43.917096 8335 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:45125 (error 108)
I20260925 13:07:43.917294 8336 catalog_manager.cc:4841] TS b540febd45d340fd86f9f8cf6f1b57db (127.0.206.67:33689): tablet 34d66081d90e4bb6b56b8f5d0cd8934f (table test-table [id=b55fc3e161f74f7787f977ca967f71d5]) successfully deleted
I20260925 13:07:43.917649 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 8561
I20260925 13:07:43.917689 8952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:43.917502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:45125 (local address 127.0.0.1:43234)
0925 13:07:43.917601 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:43.917652 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:45125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":42,"threads_started":1}
W20260925 13:07:43.919816 8335 catalog_manager.cc:4584] TS 2f14f61c0a12443eb448bbcb217b5116 (127.0.206.65:45125): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 34d66081d90e4bb6b56b8f5d0cd8934f: Network error: Client connection negotiation failed: client connection to 127.0.206.65:45125: connect: Connection refused (error 111)
I20260925 13:07:43.923247 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 8701
I20260925 13:07:43.929440 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 8317
2026-09-25T13:07:43Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2773 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:07:43 dist-test-slave-chkf krb5kdc[8955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:07:43 dist-test-slave-chkf krb5kdc[8955](info): set up 2 sockets
Sep 25 13:07:43 dist-test-slave-chkf krb5kdc[8955](info): commencing operation
krb5kdc: starting...
W20260925 13:07:44.955292 825 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:07:45 dist-test-slave-chkf krb5kdc[8955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:07:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:07:45.070003 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43683
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:43683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:45.149909 8968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:45.150146 8968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:45.150187 8968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:45.151427 8968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:45.151492 8968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:45.151520 8968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:45.151548 8968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:45.151579 8968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:07:45.153056 8968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:43683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:45.153340 8968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:45.153564 8968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:45.158401 8976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:45.158459 8968 server_base.cc:1034] running on GCE node
W20260925 13:07:45.158401 8974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:45.158422 8973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:45.158874 8968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:45.159209 8968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:45.160384 8968 hybrid_clock.cc:648] HybridClock initialized: now 1790341665160360 us; error 36 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-25T13:07:45Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1237 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:07:45 dist-test-slave-chkf krb5kdc[8981](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:07:45 dist-test-slave-chkf krb5kdc[8981](info): set up 2 sockets
Sep 25 13:07:45 dist-test-slave-chkf krb5kdc[8981](info): commencing operation
krb5kdc: starting...
W20260925 13:07:46.199149 825 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:07:46 dist-test-slave-chkf krb5kdc[8981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:07:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:07:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:07:46.311728 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:41339
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:41339
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:46.389186 8994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:46.389381 8994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:46.389402 8994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:46.390539 8994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:46.390580 8994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:46.390594 8994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:46.390607 8994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:46.390666 8994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:07:46.392181 8994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:41339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:41339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:46.392426 8994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:46.392647 8994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:46.397471 9002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:46.397536 8999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:46.397514 8994 server_base.cc:1034] running on GCE node
W20260925 13:07:46.397485 9000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:46.397842 8994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:46.398206 8994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:46.399397 8994 hybrid_clock.cc:648] HybridClock initialized: now 1790341666399394 us; error 38 us; skew 500 ppm
Sep 25 13:07:46 dist-test-slave-chkf krb5kdc[8981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341666, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:46.407657 8994 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:07:46.408095 8994 webserver.cc:460] Webserver started at http://127.0.206.126:42159/ using document root <none> and password file <none>
I20260925 13:07:46.408324 8994 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:46.408396 8994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:46.408543 8994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:46.409569 8994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "9d060887a3f7442a86f50a94afefb0a5"
format_stamp: "Formatted at 2026-09-25 13:07:46 on dist-test-slave-chkf"
server_key: "d42d15597d3332416b894dae6dff58ec"
server_key_iv: "ed6b5864a5b5406e03da4c423a5a8352"
server_key_version: "encryptionkey@0"
I20260925 13:07:46.409925 8994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "9d060887a3f7442a86f50a94afefb0a5"
format_stamp: "Formatted at 2026-09-25 13:07:46 on dist-test-slave-chkf"
server_key: "d42d15597d3332416b894dae6dff58ec"
server_key_iv: "ed6b5864a5b5406e03da4c423a5a8352"
server_key_version: "encryptionkey@0"
I20260925 13:07:46.411311 8994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:07:46.412014 9009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:46.412184 8994 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:46.412285 8994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "9d060887a3f7442a86f50a94afefb0a5"
format_stamp: "Formatted at 2026-09-25 13:07:46 on dist-test-slave-chkf"
server_key: "d42d15597d3332416b894dae6dff58ec"
server_key_iv: "ed6b5864a5b5406e03da4c423a5a8352"
server_key_version: "encryptionkey@0"
I20260925 13:07:46.412364 8994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:46.422606 8994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:46.424937 8994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:46.425096 8994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:46.429494 8994 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:41339
I20260925 13:07:46.429523 9061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:41339 every 8 connection(s)
I20260925 13:07:46.430006 8994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:46.430662 9062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:07:46.432945 9062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5: Bootstrap starting.
I20260925 13:07:46.433528 9062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5: Neither blocks nor log segments found. Creating new log.
I20260925 13:07:46.433820 9062 log.cc:824] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:46.434499 9062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5: No bootstrap required, opened a new log
I20260925 13:07:46.435739 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 8994
I20260925 13:07:46.435882 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:07:46.436084 825 external_mini_cluster.cc:1428] Setting key fe073f735719186b41a3678447d572c6
I20260925 13:07:46.436350 9062 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d060887a3f7442a86f50a94afefb0a5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41339 } }
I20260925 13:07:46.436506 9062 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:07:46.437108 9062 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d060887a3f7442a86f50a94afefb0a5, State: Initialized, Role: FOLLOWER
I20260925 13:07:46.437238 9062 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d060887a3f7442a86f50a94afefb0a5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41339 } }
I20260925 13:07:46.437325 9062 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:46.437350 9062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:46.437418 9062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:07:46.437924 9062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d060887a3f7442a86f50a94afefb0a5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41339 } }
I20260925 13:07:46.438045 9062 leader_election.cc:302] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d060887a3f7442a86f50a94afefb0a5; no voters:
I20260925 13:07:46.438239 9062 leader_election.cc:288] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:07:46.438262 9067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:07:46.438429 9067 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [term 1 LEADER]: Becoming Leader. State: Replica: 9d060887a3f7442a86f50a94afefb0a5, State: Running, Role: LEADER
I20260925 13:07:46.438503 9062 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:46.438611 9067 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d060887a3f7442a86f50a94afefb0a5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41339 } }
I20260925 13:07:46.438995 9067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d060887a3f7442a86f50a94afefb0a5. Latest consensus state: current_term: 1 leader_uuid: "9d060887a3f7442a86f50a94afefb0a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d060887a3f7442a86f50a94afefb0a5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41339 } } }
I20260925 13:07:46.439088 9067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:46.439236 9069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d060887a3f7442a86f50a94afefb0a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d060887a3f7442a86f50a94afefb0a5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41339 } } }
I20260925 13:07:46.439319 9069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:46.439419 9076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:46.439937 9076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:07:46.442360 9076 catalog_manager.cc:1347] Generated new cluster ID: e5adf0a58b7b43dcb25e680e6cfd1cea
I20260925 13:07:46.442410 9076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 13:07:46 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:46.447075 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:46.436589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41420 (local address 127.0.206.126:41339)
0925 13:07:46.437177 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:46.437186 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:46.437218 (+ 32us) server_negotiation.cc:407] Connection header received
0925 13:07:46.437368 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:46.437376 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:46.437429 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:46.437559 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:46.438296 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:46.439689 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:46.440677 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:46.441137 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:46.441272 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:46.443850 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:46.443853 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:46.443853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:46.443864 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:46.445524 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:46.445907 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:46.445910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:46.445910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:46.446010 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:46.446225 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:46.446226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:46.446226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:46.446455 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:46.446631 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:46.446695 (+ 64us) server_negotiation.cc:299] Negotiation successful
0925 13:07:46.446942 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":439,"threads_started":1}
I20260925 13:07:46.468276 9076 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:07:46.468765 9076 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:46.472568 9076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5: Generated new TSK 0
I20260925 13:07:46.472733 9076 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260925 13:07:46.505540 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:41339
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:46.584092 9088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:46.584283 9088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:46.584307 9088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:46.585454 9088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:46.585493 9088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:46.585520 9088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:07:46.587052 9088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:41339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:46.587284 9088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:46.587512 9088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:46.592448 9096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:46.592429 9093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:46.592448 9094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:46.592486 9088 server_base.cc:1034] running on GCE node
I20260925 13:07:46.592806 9088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:46.593070 9088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:46.594214 9088 hybrid_clock.cc:648] HybridClock initialized: now 1790341666594192 us; error 36 us; skew 500 ppm
Sep 25 13:07:46 dist-test-slave-chkf krb5kdc[8981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341666, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:46.596662 9088 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:07:46.597051 9088 webserver.cc:460] Webserver started at http://127.0.206.65:44501/ using document root <none> and password file <none>
I20260925 13:07:46.597285 9088 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:46.597349 9088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:46.597486 9088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:46.598430 9088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "9d8016e1821840d0beaa235f18584329"
format_stamp: "Formatted at 2026-09-25 13:07:46 on dist-test-slave-chkf"
server_key: "04aed141b3f8e3b165ece6896f0893ec"
server_key_iv: "468649a97d50bae9b0600d2b26072c2a"
server_key_version: "encryptionkey@0"
I20260925 13:07:46.598832 9088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "9d8016e1821840d0beaa235f18584329"
format_stamp: "Formatted at 2026-09-25 13:07:46 on dist-test-slave-chkf"
server_key: "04aed141b3f8e3b165ece6896f0893ec"
server_key_iv: "468649a97d50bae9b0600d2b26072c2a"
server_key_version: "encryptionkey@0"
I20260925 13:07:46.600144 9088 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:46.600862 9103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:46.601034 9088 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:46.601126 9088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "9d8016e1821840d0beaa235f18584329"
format_stamp: "Formatted at 2026-09-25 13:07:46 on dist-test-slave-chkf"
server_key: "04aed141b3f8e3b165ece6896f0893ec"
server_key_iv: "468649a97d50bae9b0600d2b26072c2a"
server_key_version: "encryptionkey@0"
I20260925 13:07:46.601208 9088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:46.619616 9088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:46.621745 9088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:46.621906 9088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:46.622113 9088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:46.622387 9088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:46.622439 9088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:46.622481 9088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:46.622509 9088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:46.627965 9088 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:33559
I20260925 13:07:46.628007 9216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:33559 every 8 connection(s)
I20260925 13:07:46.628278 9088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:07:46.629841 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 9088
I20260925 13:07:46.629967 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:07:46.630182 825 external_mini_cluster.cc:1428] Setting key 2e84fb6b99d2c99b4fc6cca34522b9c6
Sep 25 13:07:46 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341666, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:46.638188 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:46.628879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:49329 (local address 127.0.206.126:41339)
0925 13:07:46.628960 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:46.628963 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:46.629120 (+ 157us) server_negotiation.cc:407] Connection header received
0925 13:07:46.630105 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:46.630106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:46.630127 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:46.630178 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:46.630940 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:46.631411 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:46.632694 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:46.633028 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:46.633105 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:46.635501 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:46.635502 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:46.635502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:46.635508 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:46.636840 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:46.637287 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:46.637288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:46.637289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:46.637335 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:46.637612 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:46.637613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:46.637613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:46.637704 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:46.637752 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:46.638071 (+ 319us) server_negotiation.cc:299] Negotiation successful
0925 13:07:46.638128 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
I20260925 13:07:46.638499 9219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:46.628855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:49329)
0925 13:07:46.629003 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:46.629035 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:46.630009 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:46.630247 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:46.630251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:46.630577 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:46.630843 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:46.630868 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:46.631497 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:46.631498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:46.632613 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:46.632615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:46.633121 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:46.633122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:46.633217 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:46.633692 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:46.633700 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:46.635402 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:46.636924 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:46.636931 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:46.636933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:46.637214 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:46.637411 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:46.637412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:46.637413 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:46.637547 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:46.637775 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:46.637782 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:46.637937 (+ 155us) client_negotiation.cc:769] Sending connection context
0925 13:07:46.638053 (+ 116us) client_negotiation.cc:240] Negotiation successful
0925 13:07:46.638195 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":96,"threads_started":1}
I20260925 13:07:46.638983 9217 heartbeater.cc:347] Connected to a master server at 127.0.206.126:41339
I20260925 13:07:46.639055 9217 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:46.639248 9217 heartbeater.cc:511] Master 127.0.206.126:41339 requested a full tablet report, sending...
I20260925 13:07:46.639731 9026 ts_manager.cc:194] Registered new tserver with Master: 9d8016e1821840d0beaa235f18584329 (127.0.206.65:33559)
I20260925 13:07:46.640338 9026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:49329
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260925 13:07:46.653997 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:41339
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:46.731490 9222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:46.731673 9222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:46.731694 9222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:46.732930 9222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:46.732972 9222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:46.733000 9222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:07:46.734496 9222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:41339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:46.734753 9222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:46.734977 9222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:46.739873 9227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:46.739871 9230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:46.739919 9222 server_base.cc:1034] running on GCE node
W20260925 13:07:46.739871 9228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:46.740237 9222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:46.740483 9222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:46.741652 9222 hybrid_clock.cc:648] HybridClock initialized: now 1790341666741642 us; error 25 us; skew 500 ppm
Sep 25 13:07:46 dist-test-slave-chkf krb5kdc[8981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341666, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:46.744041 9222 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:07:46.744421 9222 webserver.cc:460] Webserver started at http://127.0.206.66:43281/ using document root <none> and password file <none>
I20260925 13:07:46.744604 9222 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:46.744642 9222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:46.744728 9222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:46.745842 9222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "86cab035a5234bbb82b43d08caa23a3c"
format_stamp: "Formatted at 2026-09-25 13:07:46 on dist-test-slave-chkf"
server_key: "6d5f00c8892a5f66fcce78cef969bac0"
server_key_iv: "1b935c27e75e554373036f451d7216b2"
server_key_version: "encryptionkey@0"
I20260925 13:07:46.746130 9222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "86cab035a5234bbb82b43d08caa23a3c"
format_stamp: "Formatted at 2026-09-25 13:07:46 on dist-test-slave-chkf"
server_key: "6d5f00c8892a5f66fcce78cef969bac0"
server_key_iv: "1b935c27e75e554373036f451d7216b2"
server_key_version: "encryptionkey@0"
I20260925 13:07:46.747418 9222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:07:46.748091 9237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:46.748253 9222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:46.748329 9222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "86cab035a5234bbb82b43d08caa23a3c"
format_stamp: "Formatted at 2026-09-25 13:07:46 on dist-test-slave-chkf"
server_key: "6d5f00c8892a5f66fcce78cef969bac0"
server_key_iv: "1b935c27e75e554373036f451d7216b2"
server_key_version: "encryptionkey@0"
I20260925 13:07:46.748385 9222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:46.766614 9222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:46.768801 9222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:46.768903 9222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:46.769057 9222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:46.769414 9222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:46.769447 9222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:46.769467 9222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:46.769515 9222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:46.774887 9222 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:36023
I20260925 13:07:46.774958 9350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:36023 every 8 connection(s)
I20260925 13:07:46.775247 9222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:07:46.778028 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 9222
I20260925 13:07:46.778146 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:07:46.778324 825 external_mini_cluster.cc:1428] Setting key 47752ae2a300754cd6e452e4d34390ea
Sep 25 13:07:46 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341666, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:46.784878 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:46.775988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:60107 (local address 127.0.206.126:41339)
0925 13:07:46.776074 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:46.776076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:46.776249 (+ 173us) server_negotiation.cc:407] Connection header received
0925 13:07:46.777141 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:46.777142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:46.777152 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:46.777203 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:46.777731 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:46.778313 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:46.779511 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:46.779723 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:46.779806 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:46.782040 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:46.782041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:46.782041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:46.782048 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:46.783531 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:46.783944 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:46.783946 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:46.783946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:46.784008 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:46.784249 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:46.784250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:46.784250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:46.784367 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:46.784444 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:46.784698 (+ 254us) server_negotiation.cc:299] Negotiation successful
0925 13:07:46.784793 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 13:07:46.785043 9353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:46.775946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:60107)
0925 13:07:46.776106 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:46.776137 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:46.777048 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:46.777254 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:46.777258 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:46.777485 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:46.777658 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:46.777661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:46.778408 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:46.778409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:46.779410 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:46.779412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:46.779815 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:46.779816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:46.779943 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:46.780436 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:46.780448 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:46.781944 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:46.783625 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:46.783631 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:46.783633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:46.783854 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:46.784075 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:46.784076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:46.784077 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:46.784179 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:46.784434 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:46.784440 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:46.784571 (+ 131us) client_negotiation.cc:769] Sending connection context
0925 13:07:46.784677 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 13:07:46.784824 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":103,"threads_started":1}
I20260925 13:07:46.785475 9351 heartbeater.cc:347] Connected to a master server at 127.0.206.126:41339
I20260925 13:07:46.785578 9351 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:46.785737 9351 heartbeater.cc:511] Master 127.0.206.126:41339 requested a full tablet report, sending...
I20260925 13:07:46.786152 9026 ts_manager.cc:194] Registered new tserver with Master: 86cab035a5234bbb82b43d08caa23a3c (127.0.206.66:36023)
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
I20260925 13:07:46.786536 9026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:60107
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260925 13:07:46.803750 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:41339
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:46.882665 9356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:46.882920 9356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:46.882942 9356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:46.884039 9356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:46.884074 9356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:46.884099 9356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:07:46.885474 9356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:41339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:46.885643 9356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:46.885804 9356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:46.890586 9364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:46.890666 9361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:46.890697 9362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:46.890806 9356 server_base.cc:1034] running on GCE node
I20260925 13:07:46.891022 9356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:46.891261 9356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:46.892398 9356 hybrid_clock.cc:648] HybridClock initialized: now 1790341666892371 us; error 45 us; skew 500 ppm
Sep 25 13:07:46 dist-test-slave-chkf krb5kdc[8981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341666, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:46.894948 9356 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:07:46.895382 9356 webserver.cc:460] Webserver started at http://127.0.206.67:44929/ using document root <none> and password file <none>
I20260925 13:07:46.895572 9356 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:46.895612 9356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:46.895696 9356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:07:46.896638 9356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "dabb079416064bea8b901320b1243d39"
format_stamp: "Formatted at 2026-09-25 13:07:46 on dist-test-slave-chkf"
server_key: "48c952d5bcaf41460a9335702fdf7c8d"
server_key_iv: "3e04869f19492519e2cc01c9c8957fc3"
server_key_version: "encryptionkey@0"
I20260925 13:07:46.896919 9356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "dabb079416064bea8b901320b1243d39"
format_stamp: "Formatted at 2026-09-25 13:07:46 on dist-test-slave-chkf"
server_key: "48c952d5bcaf41460a9335702fdf7c8d"
server_key_iv: "3e04869f19492519e2cc01c9c8957fc3"
server_key_version: "encryptionkey@0"
I20260925 13:07:46.898151 9356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:07:46.898865 9371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:46.899026 9356 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:46.899096 9356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "dabb079416064bea8b901320b1243d39"
format_stamp: "Formatted at 2026-09-25 13:07:46 on dist-test-slave-chkf"
server_key: "48c952d5bcaf41460a9335702fdf7c8d"
server_key_iv: "3e04869f19492519e2cc01c9c8957fc3"
server_key_version: "encryptionkey@0"
I20260925 13:07:46.899148 9356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:46.911777 9356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:46.913923 9356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:46.914031 9356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:46.914214 9356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:46.914561 9356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:46.914598 9356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:46.914620 9356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:46.914633 9356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:46.920368 9356 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:38403
I20260925 13:07:46.920411 9484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:38403 every 8 connection(s)
I20260925 13:07:46.920683 9356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
Sep 25 13:07:46 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341666, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:07:46.928015 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 9356
I20260925 13:07:46.928128 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:07:46.928354 825 external_mini_cluster.cc:1428] Setting key 62e378ff96856b6c20b91f5a05f556a7
I20260925 13:07:46.929695 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:46.921312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:49785 (local address 127.0.206.126:41339)
0925 13:07:46.921352 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:46.921355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:46.921536 (+ 181us) server_negotiation.cc:407] Connection header received
0925 13:07:46.922469 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:46.922470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:46.922483 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:46.922540 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:46.923213 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:46.923698 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:46.924742 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:46.924942 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:46.925018 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:46.927004 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:46.927006 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:46.927006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:46.927012 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:46.928229 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:46.928693 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:46.928694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:46.928695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:46.928779 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:46.929083 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:46.929084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:46.929085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:46.929219 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:46.929290 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:46.929574 (+ 284us) server_negotiation.cc:299] Negotiation successful
0925 13:07:46.929635 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 13:07:46.929821 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:46.921284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.67:49785)
0925 13:07:46.921411 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:46.921453 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:46.922349 (+ 896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:46.922605 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:46.922609 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:46.922936 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:46.923109 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:46.923113 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:46.923807 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:46.923809 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:46.924629 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:46.924631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:46.925022 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:46.925023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:46.925150 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:46.925549 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:46.925556 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:46.926922 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:07:46.928291 (+ 1369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:46.928297 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:46.928299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:46.928606 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:46.928852 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:07:46.928853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:07:46.928853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:07:46.928995 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:07:46.929288 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:07:46.929293 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:07:46.929460 (+ 167us) client_negotiation.cc:769] Sending connection context
0925 13:07:46.929543 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 13:07:46.929652 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":59,"threads_started":1}
I20260925 13:07:46.930231 9485 heartbeater.cc:347] Connected to a master server at 127.0.206.126:41339
I20260925 13:07:46.930316 9485 heartbeater.cc:464] Registering TS with master...
I20260925 13:07:46.930444 9485 heartbeater.cc:511] Master 127.0.206.126:41339 requested a full tablet report, sending...
I20260925 13:07:46.930796 9026 ts_manager.cc:194] Registered new tserver with Master: dabb079416064bea8b901320b1243d39 (127.0.206.67:38403)
I20260925 13:07:46.931138 9026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:49785
I20260925 13:07:46.940158 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/oryx_127.0.206.65.keytab.
Entry for principal oryx/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/oryx_127.0.206.65.keytab.
WARNING: no policy specified for oryx/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/oryx_127.0.206.66.keytab.
Entry for principal oryx/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/oryx_127.0.206.66.keytab.
WARNING: no policy specified for oryx/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/oryx_127.0.206.67.keytab.
Entry for principal oryx/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/oryx_127.0.206.67.keytab.
I20260925 13:07:47.013386 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 9088
I20260925 13:07:47.018258 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 9222
I20260925 13:07:47.022387 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 9356
I20260925 13:07:47.026952 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 8994
I20260925 13:07:47.032083 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:41339
--webserver_interface=127.0.206.126
--webserver_port=42159
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:41339
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:47.110244 9494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:47.110481 9494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:47.110518 9494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:47.111685 9494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:07:47.111760 9494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:47.111784 9494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:47.111806 9494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:07:47.111835 9494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:07:47.113315 9494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:41339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:41339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=42159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:47.113579 9494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:47.113802 9494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:47.118774 9499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:47.118791 9500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:47.118942 9494 server_base.cc:1034] running on GCE node
W20260925 13:07:47.118839 9502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:47.119242 9494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:47.119542 9494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:47.120720 9494 hybrid_clock.cc:648] HybridClock initialized: now 1790341667120714 us; error 34 us; skew 500 ppm
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341667, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:47.123257 9494 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:07:47.123682 9494 webserver.cc:460] Webserver started at http://127.0.206.126:42159/ using document root <none> and password file <none>
I20260925 13:07:47.123940 9494 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:47.124024 9494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:47.125373 9494 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:47.126119 9509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:47.126638 9494 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 13:07:47.126748 9494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "9d060887a3f7442a86f50a94afefb0a5"
format_stamp: "Formatted at 2026-09-25 13:07:46 on dist-test-slave-chkf"
server_key: "d42d15597d3332416b894dae6dff58ec"
server_key_iv: "ed6b5864a5b5406e03da4c423a5a8352"
server_key_version: "encryptionkey@0"
I20260925 13:07:47.127192 9494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:47.184510 9494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:47.186424 9494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:47.186535 9494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:47.190344 9494 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:41339
I20260925 13:07:47.190407 9561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:41339 every 8 connection(s)
I20260925 13:07:47.190994 9494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:07:47.192224 9562 sys_catalog.cc:263] Verifying existing consensus state
I20260925 13:07:47.192869 9562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5: Bootstrap starting.
I20260925 13:07:47.197093 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 9494
I20260925 13:07:47.197404 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:33559
--local_ip_for_outbound_sockets=127.0.206.65
--tserver_master_addrs=127.0.206.126:41339
--webserver_port=44501
--webserver_interface=127.0.206.65
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.206.65
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/oryx_127.0.206.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:07:47.200913 9562 log.cc:824] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5: Log is configured to *not* fsync() on all Append() calls
I20260925 13:07:47.202574 9562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:07:47.202909 9562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5: Bootstrap complete.
I20260925 13:07:47.205101 9562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d060887a3f7442a86f50a94afefb0a5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41339 } }
I20260925 13:07:47.205333 9562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d060887a3f7442a86f50a94afefb0a5, State: Initialized, Role: FOLLOWER
I20260925 13:07:47.205538 9562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d060887a3f7442a86f50a94afefb0a5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41339 } }
I20260925 13:07:47.205631 9562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:07:47.205684 9562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:07:47.205732 9562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:07:47.206678 9562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d060887a3f7442a86f50a94afefb0a5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41339 } }
I20260925 13:07:47.206826 9562 leader_election.cc:302] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d060887a3f7442a86f50a94afefb0a5; no voters:
I20260925 13:07:47.206983 9562 leader_election.cc:288] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 13:07:47.207093 9566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 13:07:47.207238 9562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:07:47.207279 9566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [term 2 LEADER]: Becoming Leader. State: Replica: 9d060887a3f7442a86f50a94afefb0a5, State: Running, Role: LEADER
I20260925 13:07:47.207463 9566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d060887a3f7442a86f50a94afefb0a5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41339 } }
I20260925 13:07:47.207777 9566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9d060887a3f7442a86f50a94afefb0a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d060887a3f7442a86f50a94afefb0a5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41339 } } }
I20260925 13:07:47.207831 9567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d060887a3f7442a86f50a94afefb0a5. Latest consensus state: current_term: 2 leader_uuid: "9d060887a3f7442a86f50a94afefb0a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d060887a3f7442a86f50a94afefb0a5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41339 } } }
I20260925 13:07:47.207950 9567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:47.207902 9566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5 [sys.catalog]: This master's current role is: LEADER
I20260925 13:07:47.208175 9575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:07:47.208658 9575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:07:47.208803 9575 catalog_manager.cc:1259] Loaded cluster ID: e5adf0a58b7b43dcb25e680e6cfd1cea
I20260925 13:07:47.208839 9575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:07:47.210032 9575 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:07:47.210201 9575 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9d060887a3f7442a86f50a94afefb0a5: Loaded TSK: 0
I20260925 13:07:47.210405 9575 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 13:07:47.278072 9564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:47.278285 9564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:47.278334 9564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:47.279525 9564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:47.279587 9564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:47.279632 9564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:07:47.281028 9564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/oryx_127.0.206.65.keytab
--principal=oryx/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:33559
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=44501
--tserver_master_addrs=127.0.206.126:41339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:47.281297 9564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:47.281514 9564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:47.286295 9588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:47.286294 9589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:47.286408 9591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:47.286550 9564 server_base.cc:1034] running on GCE node
I20260925 13:07:47.286862 9564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:47.287067 9564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:47.288203 9564 hybrid_clock.cc:648] HybridClock initialized: now 1790341667288191 us; error 26 us; skew 500 ppm
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341667, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:47.290647 9564 init.cc:377] Logged in from keytab as oryx/127.0.206.65@KRBTEST.COM (short username oryx)
I20260925 13:07:47.291088 9564 webserver.cc:460] Webserver started at http://127.0.206.65:44501/ using document root <none> and password file <none>
I20260925 13:07:47.291320 9564 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:47.291402 9564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:47.292788 9564 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:47.293390 9598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:47.293555 9564 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:47.293648 9564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "9d8016e1821840d0beaa235f18584329"
format_stamp: "Formatted at 2026-09-25 13:07:46 on dist-test-slave-chkf"
server_key: "04aed141b3f8e3b165ece6896f0893ec"
server_key_iv: "468649a97d50bae9b0600d2b26072c2a"
server_key_version: "encryptionkey@0"
I20260925 13:07:47.293982 9564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:47.314539 9564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:47.316641 9564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:47.316818 9564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:47.317040 9564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:47.317322 9564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:47.317375 9564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:47.317415 9564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:47.317442 9564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:47.323738 9564 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:33559
I20260925 13:07:47.323777 9711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:33559 every 8 connection(s)
I20260925 13:07:47.324126 9564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:47.332273 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 9564
I20260925 13:07:47.332566 9714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:47.324840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:42655)
0925 13:07:47.325011 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:47.325049 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:47.325499 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:47.325902 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:47.325904 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:47.326445 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:47.326616 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:47.326620 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.328175 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.328176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:47.328926 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:47.328928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.329611 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.329612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:47.329700 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.330200 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:47.330208 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:47.332366 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260925 13:07:47.332584 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:36023
--local_ip_for_outbound_sockets=127.0.206.66
--tserver_master_addrs=127.0.206.126:41339
--webserver_port=43281
--webserver_interface=127.0.206.66
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.206.66
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/oryx_127.0.206.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:07:47.332602 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:47.324892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:42655 (local address 127.0.206.126:41339)
0925 13:07:47.325059 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:47.325067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:47.325133 (+ 66us) server_negotiation.cc:407] Connection header received
0925 13:07:47.325672 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:47.325679 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:47.325730 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:47.325870 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:47.326736 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.328075 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.329011 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.329530 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.329631 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.332470 (+ 2839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:42655: BlockingRecv error: recv got EOF from 127.0.206.65:42655 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
W20260925 13:07:47.333209 9712 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:41339 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.206.126:41339: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:47.337965 9714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:47.333469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:49761)
0925 13:07:47.333510 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:47.333520 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:47.333552 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:47.333809 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:47.333810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:47.333991 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:47.334041 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:47.334043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.334977 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.334978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:47.335474 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:47.335475 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.335840 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.335841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:47.335903 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.336339 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:47.336343 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:47.337890 (+ 1547us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 13:07:47.338016 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:47.333518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:49761 (local address 127.0.206.126:41339)
0925 13:07:47.333671 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:47.333673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:47.333680 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:07:47.333699 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:47.333700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:47.333711 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:47.333778 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:47.334131 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.334901 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.335558 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.335757 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.335838 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.337957 (+ 2119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:49761: BlockingRecv error: recv got EOF from 127.0.206.65:49761 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:47.342938 9714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:47.338474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:52889)
0925 13:07:47.338535 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:47.338544 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:47.338579 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:47.338733 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:47.338734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:47.338885 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:47.338918 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:47.338920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.339479 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.339480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:47.340338 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:47.340340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.340654 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.340655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:47.340718 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.341050 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:47.341055 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:47.342860 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260925 13:07:47.342960 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:47.338524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:52889 (local address 127.0.206.126:41339)
0925 13:07:47.338562 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:47.338565 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:47.338572 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:07:47.338637 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:47.338638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:47.338645 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:47.338691 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:47.338991 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.339401 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.340425 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.340583 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.340630 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.342925 (+ 2295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:52889: BlockingRecv error: recv got EOF from 127.0.206.65:52889 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260925 13:07:47.343055 9712 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 13:07:47.411540 9716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:47.411728 9716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:47.411751 9716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:47.413050 9716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:47.413101 9716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:47.413129 9716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:07:47.414531 9716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/oryx_127.0.206.66.keytab
--principal=oryx/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:36023
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=43281
--tserver_master_addrs=127.0.206.126:41339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:47.414770 9716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:47.415040 9716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:47.419873 9724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:47.419938 9721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:47.419996 9716 server_base.cc:1034] running on GCE node
W20260925 13:07:47.419883 9722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:47.420264 9716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:47.420486 9716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:47.421631 9716 hybrid_clock.cc:648] HybridClock initialized: now 1790341667421615 us; error 38 us; skew 500 ppm
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341667, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:47.424044 9716 init.cc:377] Logged in from keytab as oryx/127.0.206.66@KRBTEST.COM (short username oryx)
I20260925 13:07:47.424439 9716 webserver.cc:460] Webserver started at http://127.0.206.66:43281/ using document root <none> and password file <none>
I20260925 13:07:47.424655 9716 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:47.424731 9716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:47.425947 9716 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:47.426493 9731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:47.426653 9716 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:47.426777 9716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "86cab035a5234bbb82b43d08caa23a3c"
format_stamp: "Formatted at 2026-09-25 13:07:46 on dist-test-slave-chkf"
server_key: "6d5f00c8892a5f66fcce78cef969bac0"
server_key_iv: "1b935c27e75e554373036f451d7216b2"
server_key_version: "encryptionkey@0"
I20260925 13:07:47.427084 9716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:47.443707 9716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:47.446183 9716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:47.446348 9716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:47.446575 9716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:47.446877 9716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:47.446909 9716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:47.446930 9716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:47.446943 9716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:47.452419 9716 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:36023
I20260925 13:07:47.452471 9844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:36023 every 8 connection(s)
I20260925 13:07:47.452781 9716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:07:47.457423 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 9716
I20260925 13:07:47.457752 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:38403
--local_ip_for_outbound_sockets=127.0.206.67
--tserver_master_addrs=127.0.206.126:41339
--webserver_port=44929
--webserver_interface=127.0.206.67
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.206.67
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/oryx_127.0.206.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:47.459791 9847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:47.453407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:46161)
0925 13:07:47.453552 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:47.453590 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:47.454027 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:47.454237 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:47.454240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:47.454828 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:47.455006 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:47.455011 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.455623 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.455624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:47.456377 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:47.456379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.456712 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.456713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:47.456801 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.457188 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:47.457195 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:47.459669 (+ 2474us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
W20260925 13:07:47.459892 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:47.453464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:46161 (local address 127.0.206.126:41339)
0925 13:07:47.453523 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:47.453525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:47.453673 (+ 148us) server_negotiation.cc:407] Connection header received
0925 13:07:47.454102 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:47.454103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:47.454115 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:47.454169 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:47.455080 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.455549 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.456446 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.456628 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.456694 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.459792 (+ 3098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:46161: BlockingRecv error: recv got EOF from 127.0.206.66:46161 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260925 13:07:47.460019 9845 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:41339 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.206.126:41339: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:47.464704 9847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:47.460236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:34477)
0925 13:07:47.460351 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:47.460360 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:47.460401 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:47.460516 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:47.460516 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:47.460616 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:47.460656 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:47.460657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.461542 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.461543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:47.462008 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:47.462009 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.462513 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.462514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:47.462557 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.462973 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:47.462977 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:47.464666 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95}
W20260925 13:07:47.464846 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:47.460311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:34477 (local address 127.0.206.126:41339)
0925 13:07:47.460366 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:47.460368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:47.460392 (+ 24us) server_negotiation.cc:407] Connection header received
0925 13:07:47.460453 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:47.460453 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:47.460463 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:47.460510 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:47.460718 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.461457 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.462093 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.462436 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.462512 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.464770 (+ 2258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:34477: BlockingRecv error: recv got EOF from 127.0.206.66:34477 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:47.469302 9847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:47.464975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:41315)
0925 13:07:47.465006 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:47.465015 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:47.465050 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:47.465181 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:47.465182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:47.465278 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:47.465302 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:47.465303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.466179 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.466180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:47.466626 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:47.466627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.467111 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.467112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:47.467146 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.467541 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:47.467544 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:47.469233 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":20}
W20260925 13:07:47.469362 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:47.465019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:41315 (local address 127.0.206.126:41339)
0925 13:07:47.465050 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:47.465052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:47.465086 (+ 34us) server_negotiation.cc:407] Connection header received
0925 13:07:47.465106 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:47.465106 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:47.465116 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:47.465172 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:47.465370 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.466103 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.466699 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.467039 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.467116 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.469302 (+ 2186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:41315: BlockingRecv error: recv got EOF from 127.0.206.66:41315 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260925 13:07:47.469398 9845 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 13:07:47.540681 9848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:07:47.540877 9848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:07:47.540901 9848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:07:47.542107 9848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:07:47.542152 9848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:07:47.542204 9848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:07:47.543673 9848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/krb5kdc/oryx_127.0.206.67.keytab
--principal=oryx/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:38403
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=44929
--tserver_master_addrs=127.0.206.126:41339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:07:47.543900 9848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:07:47.544114 9848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:07:47.548694 9853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:07:47.548704 9856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:47.548728 9848 server_base.cc:1034] running on GCE node
W20260925 13:07:47.548688 9854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:07:47.549234 9848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:07:47.549427 9848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:07:47.550554 9848 hybrid_clock.cc:648] HybridClock initialized: now 1790341667550545 us; error 28 us; skew 500 ppm
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341667, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:07:47.552901 9848 init.cc:377] Logged in from keytab as oryx/127.0.206.67@KRBTEST.COM (short username oryx)
I20260925 13:07:47.553282 9848 webserver.cc:460] Webserver started at http://127.0.206.67:44929/ using document root <none> and password file <none>
I20260925 13:07:47.553499 9848 fs_manager.cc:359] Metadata directory not provided
I20260925 13:07:47.553578 9848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:07:47.554821 9848 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:07:47.555379 9863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:47.555537 9848 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:07:47.555630 9848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "dabb079416064bea8b901320b1243d39"
format_stamp: "Formatted at 2026-09-25 13:07:46 on dist-test-slave-chkf"
server_key: "48c952d5bcaf41460a9335702fdf7c8d"
server_key_iv: "3e04869f19492519e2cc01c9c8957fc3"
server_key_version: "encryptionkey@0"
I20260925 13:07:47.555939 9848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:07:47.564553 9848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:07:47.566143 9848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:07:47.566291 9848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:07:47.566501 9848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:07:47.566838 9848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:07:47.566890 9848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:47.566927 9848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:07:47.566955 9848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:07:47.572525 9848 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:38403
I20260925 13:07:47.572563 9976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:38403 every 8 connection(s)
I20260925 13:07:47.572880 9848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:47.579118 9979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:47.573520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.67:45977)
0925 13:07:47.573671 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:47.573708 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:47.574124 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:47.574300 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:47.574303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:47.574898 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:47.575056 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:47.575060 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.575661 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.575662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:47.576389 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:47.576390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.576685 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.576686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:47.576770 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.577116 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:47.577122 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:47.578998 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":89,"threads_started":1}
W20260925 13:07:47.579119 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:47.573541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:45977 (local address 127.0.206.126:41339)
0925 13:07:47.573600 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:47.573602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:47.573784 (+ 182us) server_negotiation.cc:407] Connection header received
0925 13:07:47.574196 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:47.574197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:47.574206 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:47.574256 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:47.575125 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.575575 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.576454 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.576621 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.576666 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.579075 (+ 2409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:45977: BlockingRecv error: recv got EOF from 127.0.206.67:45977 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260925 13:07:47.579430 9977 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:41339 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.206.126:41339: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:47.582928 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 9848
I20260925 13:07:47.583184 9979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:47.579631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.67:36181)
0925 13:07:47.579682 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:47.579691 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:47.579750 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:47.579924 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:47.579924 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:47.580021 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:47.580056 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:47.580057 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.580574 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.580575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:47.581016 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:47.581017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.581281 (+ 264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.581282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:47.581323 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.581615 (+ 292us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:47.581619 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:47.583120 (+ 1501us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 13:07:47.583298 9715 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:47.579766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:36181 (local address 127.0.206.126:41339)
0925 13:07:47.579829 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:47.579830 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:47.579837 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:07:47.579854 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:47.579855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:47.579862 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:47.579906 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:47.580112 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.580517 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.581074 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.581228 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.581270 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.583216 (+ 1946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:36181: BlockingRecv error: recv got EOF from 127.0.206.67:36181 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:47 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:47.587934 9979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:47.583625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.67:47763)
0925 13:07:47.583655 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:47.583664 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:47.583696 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:47.583866 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:47.583866 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:47.584006 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:47.584036 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:47.584038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.584759 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.584760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:47.585404 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:47.585406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.585716 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.585717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:47.585764 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.586134 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:47.586138 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:47.587888 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":17}
W20260925 13:07:47.588059 9981 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:47.583673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:47763 (local address 127.0.206.126:41339)
0925 13:07:47.583756 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:47.583757 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:47.583763 (+ 6us) server_negotiation.cc:407] Connection header received
0925 13:07:47.583777 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:47.583777 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:47.583785 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:47.583840 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:47.584108 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.584687 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.585479 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.585650 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.585696 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.587984 (+ 2288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:47763: BlockingRecv error: recv got EOF from 127.0.206.67:47763 (error 108)
Metrics: {"server-negotiator.queue_time_us":70,"thread_start_us":55,"threads_started":1}
W20260925 13:07:47.588222 9977 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 13:07:47.590266 9715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:47.583471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41436 (local address 127.0.206.126:41339)
0925 13:07:47.583507 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:47.583509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:47.583517 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:47.583626 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:47.583627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:47.583635 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:47.583696 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:47.584398 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.584945 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.585908 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:47.586182 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:47.586229 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:47.587629 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:07:47.587634 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:07:47.587634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:07:47.587644 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:07:47.589241 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:47.589543 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:47.589545 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:47.589546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:47.589623 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:07:47.589784 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:07:47.589784 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:07:47.589784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:07:47.589970 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:07:47.590067 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 13:07:47.590146 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 13:07:47.590217 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Sep 25 13:07:48 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:48 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:48.347626 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:48.343402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:35103)
0925 13:07:48.343514 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:48.343532 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:48.343577 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:48.343696 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:48.343697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:48.343846 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:48.343893 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:48.343894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:48.344509 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:48.344510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:48.345054 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:48.345056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:48.345362 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:48.345363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:48.345400 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:48.345811 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:48.345815 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:48.347568 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":70,"threads_started":1}
W20260925 13:07:48.347767 9983 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:48.343467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:35103 (local address 127.0.206.126:41339)
0925 13:07:48.343586 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:48.343590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:48.343602 (+ 12us) server_negotiation.cc:407] Connection header received
0925 13:07:48.343620 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:48.343621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:48.343630 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:48.343686 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:48.343957 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:48.344446 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:48.345124 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:48.345301 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:48.345354 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:48.347654 (+ 2300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:35103: BlockingRecv error: recv got EOF from 127.0.206.65:35103 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
Sep 25 13:07:48 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:48 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:48.473963 9984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:48.469748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:47121)
0925 13:07:48.469866 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:48.469879 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:48.469941 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:48.470058 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:48.470058 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:48.470204 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:48.470257 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:48.470258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:48.470852 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:48.470852 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:48.471385 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:48.471387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:48.471674 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:48.471675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:48.471713 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:48.472111 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:48.472115 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:48.473860 (+ 1745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":64,"threads_started":1}
W20260925 13:07:48.473999 9983 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:48.469835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:47121 (local address 127.0.206.126:41339)
0925 13:07:48.469885 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:48.469886 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:48.469921 (+ 35us) server_negotiation.cc:407] Connection header received
0925 13:07:48.469983 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:48.469984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:48.469993 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:48.470036 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:48.470319 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:48.470794 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:48.471450 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:48.471617 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:48.471661 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:48.473939 (+ 2278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:47121: BlockingRecv error: recv got EOF from 127.0.206.66:47121 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 25 13:07:48 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:48 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:48.592782 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:48.588573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.67:33769)
0925 13:07:48.588696 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:48.588707 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:48.588765 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:48.588888 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:48.588889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:48.589054 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:48.589098 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:48.589099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:48.589680 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:48.589680 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:48.590203 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:48.590204 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:48.590519 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:48.590520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:48.590564 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:48.591003 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:48.591007 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:48.592722 (+ 1715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":81,"threads_started":1}
W20260925 13:07:48.592914 9983 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:48.588626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:33769 (local address 127.0.206.126:41339)
0925 13:07:48.588694 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:48.588697 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:48.588754 (+ 57us) server_negotiation.cc:407] Connection header received
0925 13:07:48.588818 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:48.588819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:48.588829 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:48.588868 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:48.589158 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:48.589620 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:48.590279 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:48.590457 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:48.590504 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:48.592833 (+ 2329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:33769: BlockingRecv error: recv got EOF from 127.0.206.67:33769 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 25 13:07:49 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:49 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:49.352663 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:49.348060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:35403)
0925 13:07:49.348174 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:49.348185 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:49.348243 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:49.348389 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:49.348390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:49.348537 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:49.348582 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:49.348583 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:49.349221 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:49.349222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:49.349907 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:49.349910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:49.350320 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:49.350320 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:49.350373 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:49.350881 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:49.350886 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:49.352596 (+ 1710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":72,"threads_started":1}
W20260925 13:07:49.352816 9987 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:49.348148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:35403 (local address 127.0.206.126:41339)
0925 13:07:49.348263 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:49.348265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:49.348274 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:49.348306 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:49.348307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:49.348316 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:49.348376 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:49.348651 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:49.349151 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:49.349997 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:49.350256 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:49.350312 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:49.352691 (+ 2379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:35403: BlockingRecv error: recv got EOF from 127.0.206.65:35403 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":70,"threads_started":1}
Sep 25 13:07:49 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:49 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:49.479108 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:49.474427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:35485)
0925 13:07:49.474556 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:49.474567 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:49.474620 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:49.474755 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:49.474756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:49.474913 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:49.474962 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:49.474964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:49.475588 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:49.475588 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:49.476120 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:49.476122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:49.476481 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:49.476481 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:49.476519 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:49.477022 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:49.477026 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:49.479014 (+ 1988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
W20260925 13:07:49.479154 9987 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:49.474511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:35485 (local address 127.0.206.126:41339)
0925 13:07:49.474556 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:49.474558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:49.474609 (+ 51us) server_negotiation.cc:407] Connection header received
0925 13:07:49.474675 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:49.474675 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:49.474685 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:49.474742 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:49.475037 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:49.475517 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:49.476217 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:49.476411 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:49.476473 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:49.479092 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:35485: BlockingRecv error: recv got EOF from 127.0.206.66:35485 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 25 13:07:49 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:49 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:49.597914 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:49.593214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.67:39715)
0925 13:07:49.593375 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:49.593388 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:49.593452 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:49.593633 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:49.593634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:49.593817 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:49.593862 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:49.593864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:49.594522 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:49.594524 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:49.595071 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:49.595073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:49.595457 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:49.595458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:49.595523 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:49.596016 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:49.596021 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:49.597853 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":69,"threads_started":1}
W20260925 13:07:49.597983 9987 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:49.593347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:39715 (local address 127.0.206.126:41339)
0925 13:07:49.593412 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:49.593414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:49.593431 (+ 17us) server_negotiation.cc:407] Connection header received
0925 13:07:49.593533 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:49.593534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:49.593548 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:49.593627 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:49.593949 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:49.594436 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:49.595157 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:49.595352 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:49.595457 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:49.597939 (+ 2482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:39715: BlockingRecv error: recv got EOF from 127.0.206.67:39715 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 25 13:07:50 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:50 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:50.357551 9990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:50.353069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:33027)
0925 13:07:50.353337 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:50.353349 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:50.353402 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:50.353520 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:50.353520 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:50.353673 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:50.353720 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:50.353722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:50.354353 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:50.354354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:50.354946 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:50.354948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:50.355280 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:50.355281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:50.355318 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:50.355811 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:50.355816 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:50.357497 (+ 1681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":185,"threads_started":1}
W20260925 13:07:50.357661 9991 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:50.353267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:33027 (local address 127.0.206.126:41339)
0925 13:07:50.353398 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:50.353400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:50.353411 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:07:50.353438 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:50.353438 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:50.353448 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:50.353505 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:50.353785 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:50.354272 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:50.355020 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:50.355218 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:50.355271 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:50.357582 (+ 2311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:33027: BlockingRecv error: recv got EOF from 127.0.206.65:33027 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
Sep 25 13:07:50 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:50 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:50.484009 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:50.479569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:44015)
0925 13:07:50.479738 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:50.479748 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:50.479803 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:50.479939 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:50.479940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:50.480098 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:50.480148 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:50.480149 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:50.480761 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:50.480762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:50.481285 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:50.481287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:50.481596 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:50.481597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:50.481639 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:50.482077 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:50.482082 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:50.483942 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":96,"threads_started":1}
W20260925 13:07:50.484076 9991 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:50.479642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:44015 (local address 127.0.206.126:41339)
0925 13:07:50.479709 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:50.479711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:50.479804 (+ 93us) server_negotiation.cc:407] Connection header received
0925 13:07:50.479865 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:50.479866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:50.479877 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:50.479917 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:50.480207 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:50.480684 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:50.481357 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:50.481534 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:50.481582 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:50.484032 (+ 2450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:44015: BlockingRecv error: recv got EOF from 127.0.206.66:44015 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 25 13:07:50 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:50 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:50.602697 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:50.598480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.67:50661)
0925 13:07:50.598644 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:50.598656 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:50.598741 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:50.598883 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:50.598884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:50.599038 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:50.599085 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:50.599086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:50.599678 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:50.599679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:50.600170 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:50.600171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:50.600483 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:50.600483 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:50.600518 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:50.601001 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:50.601007 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:50.602644 (+ 1637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":76,"threads_started":1}
W20260925 13:07:50.602859 9991 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:50.598662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:50661 (local address 127.0.206.126:41339)
0925 13:07:50.598743 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:50.598744 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:50.598780 (+ 36us) server_negotiation.cc:407] Connection header received
0925 13:07:50.598800 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:50.598800 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:50.598811 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:50.598850 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:50.599158 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:50.599619 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:50.600233 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:50.600408 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:50.600474 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:50.602782 (+ 2308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:50661: BlockingRecv error: recv got EOF from 127.0.206.67:50661 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 25 13:07:51 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:51 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:51.362656 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:51.358048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:34129)
0925 13:07:51.358180 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:51.358196 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:51.358243 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:51.358392 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:51.358393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:51.358554 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:51.358600 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:51.358602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:51.359391 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:51.359392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:51.360079 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:51.360081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:51.360393 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:51.360394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:51.360432 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:51.360916 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:51.360920 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:51.362599 (+ 1679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
W20260925 13:07:51.362828 9995 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:51.358096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:34129 (local address 127.0.206.126:41339)
0925 13:07:51.358226 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:51.358228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:51.358237 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:51.358289 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:51.358289 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:51.358299 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:51.358364 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:51.358687 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:51.359279 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:51.360144 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:51.360330 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:51.360382 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:51.362685 (+ 2303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:34129: BlockingRecv error: recv got EOF from 127.0.206.65:34129 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
Sep 25 13:07:51 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:51 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:51.492687 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:51.484579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:59043)
0925 13:07:51.484746 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:51.484767 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:51.484857 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:51.485026 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:51.485027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:51.485196 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:51.485239 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:51.485241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:51.485887 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:51.485888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:51.486435 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:51.486437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:51.486911 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:51.486912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:51.486975 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:51.490898 (+ 3923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:51.490902 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:51.492597 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":73,"threads_started":1}
W20260925 13:07:51.492735 9995 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:51.484673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:59043 (local address 127.0.206.126:41339)
0925 13:07:51.484750 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:51.484752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:51.484838 (+ 86us) server_negotiation.cc:407] Connection header received
0925 13:07:51.484941 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:51.484941 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:51.484953 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:51.485018 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:51.485322 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:51.485806 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:51.486572 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:51.486826 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:51.486911 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:51.492671 (+ 5760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:59043: BlockingRecv error: recv got EOF from 127.0.206.66:59043 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 25 13:07:51 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:51 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:51.608013 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:51.603194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.67:48755)
0925 13:07:51.603360 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:51.603371 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:51.603452 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:51.603581 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:51.603582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:51.603752 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:51.603794 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:51.603796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:51.604454 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:51.604455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:51.604967 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:51.604969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:51.605321 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:51.605322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:51.605389 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:51.606005 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:51.606010 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:51.607901 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
W20260925 13:07:51.608065 9995 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:51.603268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:48755 (local address 127.0.206.126:41339)
0925 13:07:51.603334 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:51.603336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:51.603433 (+ 97us) server_negotiation.cc:407] Connection header received
0925 13:07:51.603513 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:51.603514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:51.603528 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:51.603578 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:51.603878 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:51.604370 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:51.605044 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:51.605239 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:51.605320 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:51.607999 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:48755: BlockingRecv error: recv got EOF from 127.0.206.67:48755 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 25 13:07:52 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:52 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:52.367487 9998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:52.363110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:39053)
0925 13:07:52.363236 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:52.363246 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:52.363292 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:52.363430 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:52.363431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:52.363585 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:52.363633 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:52.363635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:52.364240 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:52.364241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:52.364745 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:52.364747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:52.365073 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:52.365074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:52.365111 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:52.365649 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:52.365654 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:52.367431 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
W20260925 13:07:52.367610 9999 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:52.363182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:39053 (local address 127.0.206.126:41339)
0925 13:07:52.363294 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:52.363296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:52.363337 (+ 41us) server_negotiation.cc:407] Connection header received
0925 13:07:52.363357 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:52.363357 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:52.363366 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:52.363418 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:52.363695 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:52.364180 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:52.364818 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:52.365000 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:52.365048 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:52.367523 (+ 2475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:39053: BlockingRecv error: recv got EOF from 127.0.206.65:39053 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":65,"threads_started":1}
Sep 25 13:07:52 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:52 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:52.497751 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:52.493151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:38879)
0925 13:07:52.493285 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:52.493303 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:52.493351 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:52.493473 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:52.493473 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:52.493690 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:52.493737 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:52.493739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:52.494375 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:52.494376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:52.495035 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:52.495037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:52.495334 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:52.495335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:52.495373 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:52.495773 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:52.495778 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:52.497686 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
W20260925 13:07:52.497828 9999 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:52.493250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:38879 (local address 127.0.206.126:41339)
0925 13:07:52.493326 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:52.493328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:52.493347 (+ 19us) server_negotiation.cc:407] Connection header received
0925 13:07:52.493408 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:52.493409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:52.493419 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:52.493470 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:52.493803 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:52.494295 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:52.495101 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:52.495275 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:52.495326 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:52.497783 (+ 2457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:38879: BlockingRecv error: recv got EOF from 127.0.206.66:38879 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 25 13:07:52 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:52 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:52.612905 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:52.608572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.67:37433)
0925 13:07:52.608732 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:52.608743 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:52.608802 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:52.608943 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:52.608943 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:52.609095 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:52.609139 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:52.609140 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:52.609757 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:52.609757 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:52.610262 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:52.610263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:52.610564 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:52.610564 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:52.610601 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:52.611101 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:52.611106 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:52.612849 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
W20260925 13:07:52.613049 9999 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:52.608615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:37433 (local address 127.0.206.126:41339)
0925 13:07:52.608678 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:52.608680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:52.608824 (+ 144us) server_negotiation.cc:407] Connection header received
0925 13:07:52.608850 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:52.608851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:52.608863 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:52.608936 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:52.609209 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:52.609694 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:52.610330 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:52.610505 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:52.610555 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:52.612955 (+ 2400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:37433: BlockingRecv error: recv got EOF from 127.0.206.67:37433 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 25 13:07:53 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:53 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:53.372371 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:53.368026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:44743)
0925 13:07:53.368181 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:53.368193 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:53.368254 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:53.368387 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:53.368388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:53.368546 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:53.368593 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:53.368594 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:53.369217 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:53.369217 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:53.369739 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:53.369740 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:53.370050 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:53.370051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:53.370088 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:53.370576 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:53.370581 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:53.372315 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":68,"threads_started":1}
W20260925 13:07:53.372512 10003 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:53.368082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:44743 (local address 127.0.206.126:41339)
0925 13:07:53.368243 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:53.368245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:53.368253 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:07:53.368308 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:53.368308 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:53.368318 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:53.368366 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:53.368655 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:53.369151 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:53.369804 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:53.369990 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:53.370039 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:53.372423 (+ 2384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:44743: BlockingRecv error: recv got EOF from 127.0.206.65:44743 (error 108)
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":92,"threads_started":1}
Sep 25 13:07:53 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:53 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:53.502559 10004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:53.498234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:41605)
0925 13:07:53.498420 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:53.498434 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:53.498495 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:53.498621 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:53.498622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:53.498797 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:53.498857 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:53.498858 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:53.499484 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:53.499485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:53.499979 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:53.499980 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:53.500309 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:53.500309 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:53.500356 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:53.500783 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:53.500788 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:53.502504 (+ 1716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
W20260925 13:07:53.502704 10003 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:53.498387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:41605 (local address 127.0.206.126:41339)
0925 13:07:53.498492 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:53.498493 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:53.498503 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:07:53.498541 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:53.498542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:53.498552 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:53.498601 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:53.498921 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:53.499420 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:53.500043 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:53.500247 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:53.500295 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:53.502586 (+ 2291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:41605: BlockingRecv error: recv got EOF from 127.0.206.66:41605 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 25 13:07:53 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:53 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:53.617805 10005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:53.613371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.67:54695)
0925 13:07:53.613510 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:53.613521 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:53.613579 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:53.613703 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:53.613704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:53.613853 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:53.613900 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:53.613901 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:53.614492 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:53.614492 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:53.615120 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:53.615122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:53.615411 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:53.615412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:53.615455 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:53.615958 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:53.615962 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:53.617712 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
W20260925 13:07:53.617846 10003 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:53.613431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:54695 (local address 127.0.206.126:41339)
0925 13:07:53.613497 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:53.613503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:53.613583 (+ 80us) server_negotiation.cc:407] Connection header received
0925 13:07:53.613631 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:53.613632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:53.613643 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:53.613690 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:53.613963 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:53.614430 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:53.615188 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:53.615350 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:53.615396 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:53.617786 (+ 2390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:54695: BlockingRecv error: recv got EOF from 127.0.206.67:54695 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 25 13:07:54 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:54 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:54.377390 10006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:54.372804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:58365)
0925 13:07:54.372951 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:54.372961 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:54.373005 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:54.373253 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:54.373253 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:54.373409 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:54.373455 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:54.373456 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:54.374106 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:54.374108 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:54.374643 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:54.374645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:54.375037 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:54.375038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:54.375083 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:54.375530 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:54.375546 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:54.377324 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
W20260925 13:07:54.377501 10007 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:54.372984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:58365 (local address 127.0.206.126:41339)
0925 13:07:54.373123 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:54.373129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:54.373138 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:07:54.373167 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:54.373167 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:54.373176 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:54.373237 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:54.373518 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:54.374016 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:54.374733 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:54.374966 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:54.375017 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:54.377424 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:58365: BlockingRecv error: recv got EOF from 127.0.206.65:58365 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
Sep 25 13:07:54 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:54 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:54.507306 10008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:54.502992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:41815)
0925 13:07:54.503171 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:54.503186 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:54.503243 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:54.503360 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:54.503361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:54.503510 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:54.503558 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:54.503559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:54.504160 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:54.504160 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:54.504659 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:54.504661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:54.504953 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:54.504954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:54.504989 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:54.505412 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:54.505416 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:54.507213 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":101,"threads_started":1}
W20260925 13:07:54.507359 10007 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:54.503083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:41815 (local address 127.0.206.126:41339)
0925 13:07:54.503143 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:54.503145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:54.503246 (+ 101us) server_negotiation.cc:407] Connection header received
0925 13:07:54.503284 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:54.503285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:54.503294 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:54.503344 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:54.503615 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:54.504097 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:54.504723 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:54.504894 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:54.504949 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:54.507286 (+ 2337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:41815: BlockingRecv error: recv got EOF from 127.0.206.66:41815 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 25 13:07:54 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:54 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:54.622706 10009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:54.618247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.67:55669)
0925 13:07:54.618417 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:54.618429 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:54.618516 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:54.618649 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:54.618650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:54.618834 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:54.618879 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:54.618881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:54.619546 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:54.619548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:54.620074 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:54.620076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:54.620443 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:54.620444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:54.620507 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:54.620940 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:54.620944 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:54.622616 (+ 1672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":81,"threads_started":1}
W20260925 13:07:54.622786 10007 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:54.618337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:55669 (local address 127.0.206.126:41339)
0925 13:07:54.618417 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:54.618420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:54.618498 (+ 78us) server_negotiation.cc:407] Connection header received
0925 13:07:54.618571 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:54.618572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:54.618586 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:54.618632 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:54.618959 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:54.619463 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:54.620159 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:54.620353 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:54.620443 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:54.622693 (+ 2250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:55669: BlockingRecv error: recv got EOF from 127.0.206.67:55669 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 25 13:07:55 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:55 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:55.382479 10010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:55.377895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:59709)
0925 13:07:55.378051 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:55.378069 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:55.378115 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:55.378247 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:55.378248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:55.378403 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:55.378449 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:55.378451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:55.379163 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:55.379164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:55.379699 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:55.379701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:55.380069 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:55.380070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:55.380107 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:55.380623 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:55.380628 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:55.382415 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
W20260925 13:07:55.382578 10011 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:55.378007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:59709 (local address 127.0.206.126:41339)
0925 13:07:55.378134 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:55.378136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:55.378143 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:07:55.378163 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:55.378163 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:55.378172 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:55.378230 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:55.378537 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:55.379089 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:55.379803 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:55.379990 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:55.380043 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:55.382510 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:59709: BlockingRecv error: recv got EOF from 127.0.206.65:59709 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
Sep 25 13:07:55 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:55 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:55.512454 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:55.507782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:52123)
0925 13:07:55.507949 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:55.507961 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:55.508036 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:55.508194 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:55.508195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:55.508390 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:55.508481 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:55.508484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:55.509257 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:55.509258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:55.509795 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:55.509796 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:55.510109 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:55.510109 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:55.510147 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:55.510583 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:55.510588 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:55.512351 (+ 1763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
W20260925 13:07:55.512483 10011 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:55.507877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:52123 (local address 127.0.206.126:41339)
0925 13:07:55.507939 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:55.507958 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:55.508016 (+ 58us) server_negotiation.cc:407] Connection header received
0925 13:07:55.508119 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:55.508119 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:55.508129 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:55.508176 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:55.508559 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:55.509184 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:55.509866 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:55.510048 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:55.510095 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:55.512426 (+ 2331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:52123: BlockingRecv error: recv got EOF from 127.0.206.66:52123 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 25 13:07:55 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:55 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:55.627605 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:55.623186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.67:53629)
0925 13:07:55.623328 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:55.623341 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:55.623400 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:55.623600 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:55.623600 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:55.623753 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:55.623801 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:55.623802 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:55.624424 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:55.624425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:55.624945 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:55.624946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:55.625241 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:55.625241 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:55.625277 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:55.625789 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:55.625800 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:55.627551 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
W20260925 13:07:55.627712 10011 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:55.623221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:53629 (local address 127.0.206.126:41339)
0925 13:07:55.623312 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:55.623315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:55.623401 (+ 86us) server_negotiation.cc:407] Connection header received
0925 13:07:55.623512 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:55.623513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:55.623529 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:55.623580 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:55.623860 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:55.624355 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:55.625015 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:55.625183 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:55.625229 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:55.627631 (+ 2402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:53629: BlockingRecv error: recv got EOF from 127.0.206.67:53629 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 25 13:07:56 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:56 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:56.387825 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:56.383049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:51527)
0925 13:07:56.383186 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:56.383196 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:56.383255 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:56.383411 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:56.383411 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:56.383567 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:56.383614 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:56.383616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:56.384234 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:56.384234 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:56.384956 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:56.384959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:56.385348 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:56.385349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:56.385390 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:56.385822 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:56.385827 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:56.387729 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
W20260925 13:07:56.387867 10015 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:56.383116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:51527 (local address 127.0.206.126:41339)
0925 13:07:56.383248 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:56.383254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:56.383264 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:07:56.383302 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:56.383302 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:56.383312 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:56.383390 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:56.383676 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:56.384169 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:56.385063 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:56.385282 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:56.385332 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:56.387802 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:51527: BlockingRecv error: recv got EOF from 127.0.206.65:51527 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
Sep 25 13:07:56 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:56 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:56.522617 10016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:56.512927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:43647)
0925 13:07:56.513061 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:56.513071 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:56.513151 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:56.513309 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:56.513310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:56.513567 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:56.513627 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:56.513630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:56.514318 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:56.514319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:56.514881 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:56.514888 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:56.515242 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:56.515243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:56.515282 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:56.520741 (+ 5459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:56.520745 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:56.522559 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
W20260925 13:07:56.522768 10015 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:56.512991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:43647 (local address 127.0.206.126:41339)
0925 13:07:56.513073 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:56.513075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:56.513149 (+ 74us) server_negotiation.cc:407] Connection header received
0925 13:07:56.513206 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:56.513207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:56.513227 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:56.513291 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:56.513713 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:56.514230 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:56.514989 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:56.515173 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:56.515233 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:56.522657 (+ 7424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:43647: BlockingRecv error: recv got EOF from 127.0.206.66:43647 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 25 13:07:56 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:56 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:56.632867 10017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:56.628107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.67:56051)
0925 13:07:56.628284 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:56.628293 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:56.628349 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:56.628506 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:56.628507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:56.628660 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:56.628707 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:56.628708 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:56.629435 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:56.629436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:56.629954 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:56.629956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:56.630310 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:56.630311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:56.630352 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:56.630794 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:56.630800 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:56.632812 (+ 2012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":90,"threads_started":1}
W20260925 13:07:56.632992 10015 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:56.628161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:56051 (local address 127.0.206.126:41339)
0925 13:07:56.628228 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:56.628230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:56.628351 (+ 121us) server_negotiation.cc:407] Connection header received
0925 13:07:56.628396 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:56.628397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:56.628409 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:56.628462 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:56.628799 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:56.629367 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:56.630020 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:56.630259 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:56.630308 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:56.632912 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:56051: BlockingRecv error: recv got EOF from 127.0.206.67:56051 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 25 13:07:57 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:57 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:57.392761 10018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:57.388332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:37125)
0925 13:07:57.388472 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:57.388482 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:57.388528 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:57.388657 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:57.388658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:57.388819 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:57.388866 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:57.388867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:57.389477 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:57.389477 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:57.389989 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:57.389991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:57.390298 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:57.390299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:57.390341 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:57.390842 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:57.390847 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:57.392683 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
W20260925 13:07:57.392832 10019 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:57.388403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:37125 (local address 127.0.206.126:41339)
0925 13:07:57.388534 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:57.388536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:57.388546 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:07:57.388576 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:57.388576 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:57.388585 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:57.388645 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:57.388927 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:57.389413 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:57.390058 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:57.390236 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:57.390286 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:57.392754 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:37125: BlockingRecv error: recv got EOF from 127.0.206.65:37125 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
Sep 25 13:07:57 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:57 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:57.527853 10020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:57.523209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:60987)
0925 13:07:57.523336 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:57.523352 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:57.523421 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:57.523579 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:57.523580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:57.523767 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:57.523814 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:57.523816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:57.524497 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:57.524498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:57.525021 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:57.525022 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:57.525395 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:57.525396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:57.525463 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:57.525963 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:57.525970 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:57.527783 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
W20260925 13:07:57.527971 10019 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:57.523313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:60987 (local address 127.0.206.126:41339)
0925 13:07:57.523361 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:57.523372 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:57.523402 (+ 30us) server_negotiation.cc:407] Connection header received
0925 13:07:57.523499 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:57.523500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:57.523512 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:57.523562 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:57.523897 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:57.524406 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:57.525105 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:57.525314 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:57.525385 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:57.527888 (+ 2503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:60987: BlockingRecv error: recv got EOF from 127.0.206.66:60987 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 25 13:07:57 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:57 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:57.637616 10021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:57.633266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.67:34019)
0925 13:07:57.633403 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:57.633419 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:57.633464 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:57.633639 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:57.633641 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:57.633805 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:57.633853 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:57.633855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:57.634479 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:57.634480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:57.635023 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:57.635025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:57.635344 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:57.635345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:57.635395 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:57.635819 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:57.635823 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:57.637524 (+ 1701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
W20260925 13:07:57.637655 10019 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:57.633440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:34019 (local address 127.0.206.126:41339)
0925 13:07:57.633504 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:57.633506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:57.633513 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:07:57.633533 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:57.633534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:57.633544 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:57.633596 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:57.633926 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:57.634396 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:57.635091 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:57.635265 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:57.635317 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:57.637595 (+ 2278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:34019: BlockingRecv error: recv got EOF from 127.0.206.67:34019 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 25 13:07:58 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:58 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:58.397889 10022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:58.393230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:42713)
0925 13:07:58.393382 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:58.393392 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:58.393446 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:58.393587 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:58.393588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:58.393750 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:58.393796 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:58.393797 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:58.394436 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:58.394437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:58.394993 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:58.394995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:58.395374 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:58.395375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:58.395426 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:58.395865 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:58.395869 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:58.397776 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
W20260925 13:07:58.397935 10023 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:58.393328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:42713 (local address 127.0.206.126:41339)
0925 13:07:58.393426 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:58.393427 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:58.393437 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:07:58.393506 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:58.393507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:58.393516 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:58.393568 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:58.393862 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:58.394344 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:58.395067 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:58.395276 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:58.395335 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:58.397863 (+ 2528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:42713: BlockingRecv error: recv got EOF from 127.0.206.65:42713 (error 108)
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":49,"threads_started":1}
Sep 25 13:07:58 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:58 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:58.533272 10024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:58.528354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:57323)
0925 13:07:58.528512 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:58.528525 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:58.528592 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:58.528743 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:58.528744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:58.528947 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:58.528994 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:58.528996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:58.529798 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:58.529799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:58.530422 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:58.530424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:58.530915 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:58.530916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:58.530979 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:58.531422 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:58.531426 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:58.533165 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":67,"threads_started":1}
W20260925 13:07:58.533303 10023 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:58.528398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:57323 (local address 127.0.206.126:41339)
0925 13:07:58.528467 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:58.528469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:58.528572 (+ 103us) server_negotiation.cc:407] Connection header received
0925 13:07:58.528669 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:58.528670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:58.528683 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:58.528738 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:58.529086 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:58.529708 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:58.530588 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:58.530835 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:58.530894 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:58.533239 (+ 2345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:57323: BlockingRecv error: recv got EOF from 127.0.206.66:57323 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 25 13:07:58 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:58 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:58.642280 10025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:58.638124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.67:34747)
0925 13:07:58.638264 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:58.638274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:58.638328 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:58.638447 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:58.638447 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:58.638601 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:58.638647 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:58.638648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:58.639297 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:58.639297 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:58.639830 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:58.639831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:58.640121 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:58.640122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:58.640159 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:58.640560 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:58.640564 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:58.642214 (+ 1650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
W20260925 13:07:58.642342 10023 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:58.638175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:34747 (local address 127.0.206.126:41339)
0925 13:07:58.638233 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:58.638235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:58.638326 (+ 91us) server_negotiation.cc:407] Connection header received
0925 13:07:58.638369 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:58.638370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:58.638381 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:58.638428 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:58.638746 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:58.639234 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:58.639895 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:58.640062 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:58.640109 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:58.642298 (+ 2189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:34747: BlockingRecv error: recv got EOF from 127.0.206.67:34747 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 25 13:07:59 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:59 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:59.402935 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:59.398405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:60733)
0925 13:07:59.398553 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:59.398564 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:59.398627 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:59.398773 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:59.398774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:59.398943 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:59.398989 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:59.398990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:59.399645 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:59.399646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:59.400180 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:59.400182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:59.400547 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:59.400548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:59.400589 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:59.401027 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:59.401032 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:59.402877 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
W20260925 13:07:59.403131 10027 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:59.398450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:60733 (local address 127.0.206.126:41339)
0925 13:07:59.398577 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:59.398579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:59.398635 (+ 56us) server_negotiation.cc:407] Connection header received
0925 13:07:59.398670 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:59.398671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:59.398681 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:59.398759 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:59.399052 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:59.399548 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:59.400277 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:59.400471 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:59.400533 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:59.403053 (+ 2520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:60733: BlockingRecv error: recv got EOF from 127.0.206.65:60733 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
Sep 25 13:07:59 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:59 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:59.538399 10028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:59.533773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:41373)
0925 13:07:59.533946 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:59.533957 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:59.534031 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:59.534157 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:59.534158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:59.534327 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:59.534375 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:59.534376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:59.535048 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:59.535049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:59.535570 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:59.535571 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:59.535886 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:59.535887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:59.535926 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:59.536413 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:59.536418 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:59.538343 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":83,"threads_started":1}
W20260925 13:07:59.538530 10027 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:59.533834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:41373 (local address 127.0.206.126:41339)
0925 13:07:59.533885 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:59.533887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:59.534029 (+ 142us) server_negotiation.cc:407] Connection header received
0925 13:07:59.534076 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:59.534077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:59.534087 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:59.534135 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:59.534438 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:59.534983 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:59.535637 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:59.535823 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:59.535871 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:59.538450 (+ 2579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:41373: BlockingRecv error: recv got EOF from 127.0.206.66:41373 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Sep 25 13:07:59 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:07:59 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:07:59.647852 10029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:07:59.642797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.67:37687)
0925 13:07:59.642965 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 13:07:59.642975 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:07:59.643030 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:07:59.643152 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:07:59.643153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:07:59.643327 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:07:59.643381 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:59.643383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:59.644138 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:59.644140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:59.644957 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:07:59.644959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:07:59.645380 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:07:59.645382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:07:59.645427 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:59.645921 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:07:59.645926 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:07:59.647796 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
W20260925 13:07:59.648005 10027 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:07:59.642918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:37687 (local address 127.0.206.126:41339)
0925 13:07:59.642980 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 13:07:59.642982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:07:59.643017 (+ 35us) server_negotiation.cc:407] Connection header received
0925 13:07:59.643073 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:07:59.643074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:07:59.643084 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:07:59.643135 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:07:59.643484 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:59.644049 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:59.645120 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:07:59.645296 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:07:59.645356 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:07:59.647922 (+ 2566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:37687: BlockingRecv error: recv got EOF from 127.0.206.67:37687 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 25 13:08:00 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:08:00 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:08:00.408159 10030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:00.403443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:50399)
0925 13:08:00.403599 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:00.403610 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:00.403660 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:00.403972 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:00.403972 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:00.404139 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:00.404186 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:00.404188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:00.404821 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:00.404822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:00.405376 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:00.405378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:00.405691 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:00.405692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:00.405730 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:00.406178 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:08:00.406182 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:00.408043 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
W20260925 13:08:00.408217 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:08:00.403518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:50399 (local address 127.0.206.126:41339)
0925 13:08:00.403659 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:00.403661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:00.403671 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:08:00.403724 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:00.403725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:00.403736 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:00.403962 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:00.404255 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:00.404744 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:00.405446 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:00.405629 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:00.405680 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:00.408128 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:50399: BlockingRecv error: recv got EOF from 127.0.206.65:50399 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
Sep 25 13:08:00 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:08:00 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:08:00.543467 10032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:00.538926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:51055)
0925 13:08:00.539054 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:00.539069 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:00.539130 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:00.539327 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:00.539328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:00.539500 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:00.539544 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:00.539546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:00.540206 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:00.540207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:00.540716 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:00.540717 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:00.541113 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:00.541120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:00.541183 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:00.541595 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:08:00.541600 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:00.543375 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
W20260925 13:08:00.543520 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:08:00.539039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:51055 (local address 127.0.206.126:41339)
0925 13:08:00.539101 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:00.539103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:00.539111 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:08:00.539222 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:00.539222 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:00.539238 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:00.539311 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:00.539624 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:00.540125 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:00.540846 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:00.541031 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:00.541112 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:00.543456 (+ 2344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:51055: BlockingRecv error: recv got EOF from 127.0.206.66:51055 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 25 13:08:00 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:08:00 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:08:00.653087 10033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:00.648272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.67:59919)
0925 13:08:00.648572 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:00.648583 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:00.648717 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:00.648870 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:00.648871 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:00.649055 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:00.649097 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:00.649099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:00.649753 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:00.649754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:00.650302 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:00.650303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:00.650662 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:00.650663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:00.650750 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:00.651221 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:08:00.651226 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:00.652986 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":105,"threads_started":1}
W20260925 13:08:00.653118 10031 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:08:00.648361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:59919 (local address 127.0.206.126:41339)
0925 13:08:00.648553 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:00.648556 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:00.648689 (+ 133us) server_negotiation.cc:407] Connection header received
0925 13:08:00.648778 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:00.648779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:00.648794 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:00.648865 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:00.649179 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:00.649666 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:00.650395 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:00.650579 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:00.650662 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:00.653057 (+ 2395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:59919: BlockingRecv error: recv got EOF from 127.0.206.67:59919 (error 108)
Metrics: {"server-negotiator.queue_time_us":163}
Sep 25 13:08:01 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:08:01 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:08:01.413194 10034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:01.408590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:42847)
0925 13:08:01.408716 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:01.408732 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:01.408789 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:01.408964 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:01.408966 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:01.409184 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:01.409231 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:01.409232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:01.409888 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:01.409889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:01.410415 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:01.410417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:01.410766 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:01.410767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:01.410806 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:01.411278 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:08:01.411283 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:01.413096 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
W20260925 13:08:01.413241 10035 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:08:01.408658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:42847 (local address 127.0.206.126:41339)
0925 13:08:01.408794 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:01.408796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:01.408805 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:08:01.408851 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:01.408851 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:01.408861 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:01.408924 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:01.409310 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:01.409800 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:01.410488 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:01.410668 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:01.410771 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:01.413175 (+ 2404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:42847: BlockingRecv error: recv got EOF from 127.0.206.65:42847 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":52,"threads_started":1}
Sep 25 13:08:01 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:08:01 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:08:01.552033 10036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:01.543901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:43131)
0925 13:08:01.544056 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:01.544069 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:01.544137 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:01.544294 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:01.544294 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:01.544478 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:01.544522 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:01.544523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:01.545209 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:01.545210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:01.545771 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:01.545773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:01.546133 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:01.546134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:01.546202 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:01.550243 (+ 4041us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:08:01.550248 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:01.551967 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":78,"threads_started":1}
W20260925 13:08:01.552137 10035 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:08:01.543983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:43131 (local address 127.0.206.126:41339)
0925 13:08:01.544061 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:01.544064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:01.544118 (+ 54us) server_negotiation.cc:407] Connection header received
0925 13:08:01.544199 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:01.544200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:01.544213 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:01.544286 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:01.544605 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:01.545119 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:01.545857 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:01.546051 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:01.546133 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:01.552069 (+ 5936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:43131: BlockingRecv error: recv got EOF from 127.0.206.66:43131 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 25 13:08:01 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:08:01 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:08:01.658154 10037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:01.653548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.67:41513)
0925 13:08:01.653716 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:01.653728 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:01.653816 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:01.653943 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:01.653944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:01.654115 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:01.654178 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:01.654180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:01.654910 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:01.654911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:01.655440 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:01.655441 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:01.655799 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:01.655800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:01.655857 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:01.656278 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:08:01.656282 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:01.658100 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":90,"threads_started":1}
W20260925 13:08:01.658280 10035 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:08:01.653632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:41513 (local address 127.0.206.126:41339)
0925 13:08:01.653701 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:01.653703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:01.653800 (+ 97us) server_negotiation.cc:407] Connection header received
0925 13:08:01.653870 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:01.653871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:01.653884 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:01.653930 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:01.654255 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:01.654820 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:01.655517 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:01.655714 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:01.655798 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:01.658199 (+ 2401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:41513: BlockingRecv error: recv got EOF from 127.0.206.67:41513 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 25 13:08:02 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:08:02 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:08:02.418282 10038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:02.413824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.65:58597)
0925 13:08:02.414020 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:02.414031 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:02.414092 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:02.414225 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:02.414226 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:02.414383 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:02.414429 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:02.414431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:02.415135 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:02.415136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:02.415670 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:02.415671 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:02.416008 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:02.416009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:02.416058 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:02.416503 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:08:02.416508 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:02.418216 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":115,"threads_started":1}
W20260925 13:08:02.418396 10039 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:08:02.413920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:58597 (local address 127.0.206.126:41339)
0925 13:08:02.414027 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:02.414029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:02.414078 (+ 49us) server_negotiation.cc:407] Connection header received
0925 13:08:02.414144 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:02.414145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:02.414154 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:02.414201 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:02.414495 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:02.415047 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:02.415744 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:02.415925 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:02.415974 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:02.418317 (+ 2343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:58597: BlockingRecv error: recv got EOF from 127.0.206.65:58597 (error 108)
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":47,"threads_started":1}
Sep 25 13:08:02 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Sep 25 13:08:02 dist-test-slave-chkf krb5kdc[8981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260925 13:08:02.557039 10040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:02.552491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41339 (local address 127.0.206.66:53621)
0925 13:08:02.552626 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:02.552637 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:02.552698 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:02.552849 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:02.552850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:02.553034 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:02.553081 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:02.553083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:02.553742 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:02.553743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:02.554281 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:02.554282 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:02.554666 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:02.554666 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:02.554739 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:02.555175 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:08:02.555179 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:02.556985 (+ 1806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:41339: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
W20260925 13:08:02.557179 10039 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:08:02.552558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:53621 (local address 127.0.206.126:41339)
0925 13:08:02.552611 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:02.552614 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:02.552679 (+ 65us) server_negotiation.cc:407] Connection header received
0925 13:08:02.552762 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:02.552763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:02.552774 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:02.552844 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:02.553160 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:02.553659 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:02.554379 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:02.554581 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:02.554666 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:02.557101 (+ 2435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:53621: BlockingRecv error: recv got EOF from 127.0.206.66:53621 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 13:08:02.593498 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 9564
I20260925 13:08:02.598001 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 9716
I20260925 13:08:02.602126 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 9848
I20260925 13:08:02.606027 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 9494
2026-09-25T13:08:02Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17438 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-25T13:08:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:08:02Z Disabled control of system clock
I20260925 13:08:02.632602 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43433
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:34101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:43433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 13:08:02.709861 10044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:08:02.710049 10044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:08:02.710069 10044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:08:02.711179 10044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:08:02.711215 10044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:08:02.711226 10044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:08:02.711239 10044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:08:02.711251 10044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:08:02.712484 10044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:34101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:43433
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:08:02.712652 10044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:08:02.712888 10044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:08:02.717872 10052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:02.717999 10050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:02.717901 10044 server_base.cc:1034] running on GCE node
W20260925 13:08:02.717959 10049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:02.718325 10044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:08:02.718569 10044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:08:02.719724 10044 hybrid_clock.cc:648] HybridClock initialized: now 1790341682719706 us; error 33 us; skew 500 ppm
I20260925 13:08:02.720952 10044 webserver.cc:460] Webserver started at http://127.0.206.126:43775/ using document root <none> and password file <none>
I20260925 13:08:02.721194 10044 fs_manager.cc:359] Metadata directory not provided
I20260925 13:08:02.721246 10044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:08:02.721414 10044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:08:02.722388 10044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "2ab2afd35aa74f728c67d96cf666d5ac"
format_stamp: "Formatted at 2026-09-25 13:08:02 on dist-test-slave-chkf"
server_key: "7c2eb0ca8398d0272aca360c0914626a"
server_key_iv: "f783c6e5d97ce33bbb7e2d43e068591c"
server_key_version: "encryptionkey@0"
I20260925 13:08:02.722787 10044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "2ab2afd35aa74f728c67d96cf666d5ac"
format_stamp: "Formatted at 2026-09-25 13:08:02 on dist-test-slave-chkf"
server_key: "7c2eb0ca8398d0272aca360c0914626a"
server_key_iv: "f783c6e5d97ce33bbb7e2d43e068591c"
server_key_version: "encryptionkey@0"
I20260925 13:08:02.724048 10044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:08:02.724726 10058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:02.724925 10044 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:08:02.725028 10044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "2ab2afd35aa74f728c67d96cf666d5ac"
format_stamp: "Formatted at 2026-09-25 13:08:02 on dist-test-slave-chkf"
server_key: "7c2eb0ca8398d0272aca360c0914626a"
server_key_iv: "f783c6e5d97ce33bbb7e2d43e068591c"
server_key_version: "encryptionkey@0"
I20260925 13:08:02.725096 10044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:08:02.751947 10044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:08:02.752228 10044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:08:02.752369 10044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:08:02.756186 10044 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:43433
I20260925 13:08:02.756218 10110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:43433 every 8 connection(s)
I20260925 13:08:02.756609 10044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:08:02.757295 10111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:08:02.759722 10111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac: Bootstrap starting.
I20260925 13:08:02.760291 10111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac: Neither blocks nor log segments found. Creating new log.
I20260925 13:08:02.760586 10111 log.cc:824] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac: Log is configured to *not* fsync() on all Append() calls
I20260925 13:08:02.761304 10111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac: No bootstrap required, opened a new log
I20260925 13:08:02.762856 10111 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab2afd35aa74f728c67d96cf666d5ac" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43433 } }
I20260925 13:08:02.762979 10111 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:08:02.763001 10111 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab2afd35aa74f728c67d96cf666d5ac, State: Initialized, Role: FOLLOWER
I20260925 13:08:02.763161 10111 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab2afd35aa74f728c67d96cf666d5ac" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43433 } }
I20260925 13:08:02.763229 10111 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:08:02.763252 10111 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:08:02.763319 10111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:08:02.763797 10111 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab2afd35aa74f728c67d96cf666d5ac" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43433 } }
I20260925 13:08:02.763908 10111 leader_election.cc:302] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ab2afd35aa74f728c67d96cf666d5ac; no voters:
I20260925 13:08:02.764084 10111 leader_election.cc:288] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:08:02.764148 10114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:08:02.764360 10111 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:08:02.764379 10114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac [term 1 LEADER]: Becoming Leader. State: Replica: 2ab2afd35aa74f728c67d96cf666d5ac, State: Running, Role: LEADER
I20260925 13:08:02.764508 10114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab2afd35aa74f728c67d96cf666d5ac" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43433 } }
I20260925 13:08:02.764940 10115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ab2afd35aa74f728c67d96cf666d5ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab2afd35aa74f728c67d96cf666d5ac" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43433 } } }
I20260925 13:08:02.765017 10115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac [sys.catalog]: This master's current role is: LEADER
I20260925 13:08:02.765079 10116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ab2afd35aa74f728c67d96cf666d5ac. Latest consensus state: current_term: 1 leader_uuid: "2ab2afd35aa74f728c67d96cf666d5ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab2afd35aa74f728c67d96cf666d5ac" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43433 } } }
I20260925 13:08:02.765163 10116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac [sys.catalog]: This master's current role is: LEADER
I20260925 13:08:02.765463 10123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:08:02.765926 10123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:08:02.766674 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 10044
I20260925 13:08:02.766800 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:08:02.767012 825 external_mini_cluster.cc:1428] Setting key 56049ae0a9b2fa0d00e01c26233e4840
I20260925 13:08:02.771338 10123 catalog_manager.cc:1347] Generated new cluster ID: 53c2339e40c34181bcd2b3223feb38ad
I20260925 13:08:02.771395 10123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:08:02.776448 10134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:02.767424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44004 (local address 127.0.206.126:43433)
0925 13:08:02.767606 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:02.767616 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:02.771245 (+ 3629us) server_negotiation.cc:407] Connection header received
0925 13:08:02.771365 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:02.771372 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:02.771441 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:02.771586 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:02.771793 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:02.773267 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:02.774327 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:02.774984 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:02.775117 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:02.775757 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:08:02.775761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:08:02.775761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:08:02.775956 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:08:02.776010 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:08:02.776024 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:08:02.776092 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:02.776158 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 13:08:02.776230 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260925 13:08:02.784914 10123 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:08:02.785557 10123 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:08:02.797742 10123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ab2afd35aa74f728c67d96cf666d5ac: Generated new TSK 0
I20260925 13:08:02.797904 10123 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:08:02.806502 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43433
--builtin_ntp_servers=127.0.206.84:34101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 13:08:02.887436 10135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:08:02.887652 10135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:08:02.887703 10135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:08:02.888844 10135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:08:02.888911 10135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:08:02.888950 10135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:08:02.890237 10135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:34101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:08:02.890480 10135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:08:02.890774 10135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:08:02.895422 10141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:02.895447 10143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:02.895481 10140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:02.895646 10135 server_base.cc:1034] running on GCE node
I20260925 13:08:02.895845 10135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:08:02.896059 10135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:08:02.897217 10135 hybrid_clock.cc:648] HybridClock initialized: now 1790341682897204 us; error 25 us; skew 500 ppm
I20260925 13:08:02.898428 10135 webserver.cc:460] Webserver started at http://127.0.206.65:44841/ using document root <none> and password file <none>
I20260925 13:08:02.898656 10135 fs_manager.cc:359] Metadata directory not provided
I20260925 13:08:02.898767 10135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:08:02.898944 10135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:08:02.899959 10135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "665b3159bcfa4b51a77077f727daa691"
format_stamp: "Formatted at 2026-09-25 13:08:02 on dist-test-slave-chkf"
server_key: "3511facf25f4d376b6f05607e9cd946c"
server_key_iv: "f80542a428a70c6cac83bd35c643e0f4"
server_key_version: "encryptionkey@0"
I20260925 13:08:02.900303 10135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "665b3159bcfa4b51a77077f727daa691"
format_stamp: "Formatted at 2026-09-25 13:08:02 on dist-test-slave-chkf"
server_key: "3511facf25f4d376b6f05607e9cd946c"
server_key_iv: "f80542a428a70c6cac83bd35c643e0f4"
server_key_version: "encryptionkey@0"
I20260925 13:08:02.901567 10135 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:08:02.902287 10149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:02.902477 10135 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:08:02.902575 10135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "665b3159bcfa4b51a77077f727daa691"
format_stamp: "Formatted at 2026-09-25 13:08:02 on dist-test-slave-chkf"
server_key: "3511facf25f4d376b6f05607e9cd946c"
server_key_iv: "f80542a428a70c6cac83bd35c643e0f4"
server_key_version: "encryptionkey@0"
I20260925 13:08:02.902654 10135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:08:02.924937 10135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:08:02.925202 10135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:08:02.925347 10135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:08:02.925593 10135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:08:02.925933 10135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:08:02.925987 10135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:02.926019 10135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:08:02.926047 10135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:02.931972 10135 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:38415
I20260925 13:08:02.932018 10262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:38415 every 8 connection(s)
I20260925 13:08:02.932315 10135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:08:02.937280 10134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:02.932983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:60191 (local address 127.0.206.126:43433)
0925 13:08:02.933047 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:02.933049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:02.933276 (+ 227us) server_negotiation.cc:407] Connection header received
0925 13:08:02.933705 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:02.933705 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:02.933716 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:02.933770 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:02.934616 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:02.935295 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:02.936204 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:02.936380 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:02.936434 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:02.937096 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:08:02.937097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:08:02.937097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:08:02.937101 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:08:02.937113 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:08:02.937116 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:08:02.937158 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:02.937206 (+ 48us) server_negotiation.cc:299] Negotiation successful
0925 13:08:02.937226 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 13:08:02.937500 10265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:02.932975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43433 (local address 127.0.206.65:60191)
0925 13:08:02.933144 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:02.933182 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:02.933629 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:02.933820 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:02.933823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:02.933831 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:02.934507 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:02.934513 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:02.935376 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:02.935377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:02.936131 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:02.936133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:02.936450 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:02.936451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:02.936552 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:02.936976 (+ 424us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:08:02.936977 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:02.937002 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:08:02.937022 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:08:02.937158 (+ 136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:08:02.937160 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:08:02.937162 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:08:02.937207 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 13:08:02.937288 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260925 13:08:02.937938 10263 heartbeater.cc:347] Connected to a master server at 127.0.206.126:43433
I20260925 13:08:02.938033 10263 heartbeater.cc:464] Registering TS with master...
I20260925 13:08:02.938201 10263 heartbeater.cc:511] Master 127.0.206.126:43433 requested a full tablet report, sending...
I20260925 13:08:02.938678 10075 ts_manager.cc:194] Registered new tserver with Master: 665b3159bcfa4b51a77077f727daa691 (127.0.206.65:38415)
I20260925 13:08:02.939307 10075 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.65:60191
I20260925 13:08:02.941453 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 10135
I20260925 13:08:02.941529 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:08:02.941679 825 external_mini_cluster.cc:1428] Setting key 1f3bd0e50fdef95c9cda7c2dc3e7be46
I20260925 13:08:02.942483 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43433
--builtin_ntp_servers=127.0.206.84:34101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 13:08:03.020977 10266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:08:03.021189 10266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:08:03.021225 10266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:08:03.022353 10266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:08:03.022413 10266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:08:03.022452 10266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:08:03.023790 10266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:34101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:08:03.024029 10266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:08:03.024292 10266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:08:03.029217 10272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:03.029268 10266 server_base.cc:1034] running on GCE node
W20260925 13:08:03.029232 10271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:03.029233 10274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:03.029593 10266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:08:03.029778 10266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:08:03.030915 10266 hybrid_clock.cc:648] HybridClock initialized: now 1790341683030888 us; error 34 us; skew 500 ppm
I20260925 13:08:03.031953 10266 webserver.cc:460] Webserver started at http://127.0.206.66:43475/ using document root <none> and password file <none>
I20260925 13:08:03.032128 10266 fs_manager.cc:359] Metadata directory not provided
I20260925 13:08:03.032168 10266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:08:03.032251 10266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:08:03.033174 10266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "5beda895a5b24df0a09075131547ec1d"
format_stamp: "Formatted at 2026-09-25 13:08:03 on dist-test-slave-chkf"
server_key: "b7480f928f88ddc480dd31a0d3c75255"
server_key_iv: "147fde4a9e5933637d1be22e50319f97"
server_key_version: "encryptionkey@0"
I20260925 13:08:03.033453 10266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "5beda895a5b24df0a09075131547ec1d"
format_stamp: "Formatted at 2026-09-25 13:08:03 on dist-test-slave-chkf"
server_key: "b7480f928f88ddc480dd31a0d3c75255"
server_key_iv: "147fde4a9e5933637d1be22e50319f97"
server_key_version: "encryptionkey@0"
I20260925 13:08:03.034629 10266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:08:03.035382 10280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:03.035558 10266 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:08:03.035643 10266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "5beda895a5b24df0a09075131547ec1d"
format_stamp: "Formatted at 2026-09-25 13:08:03 on dist-test-slave-chkf"
server_key: "b7480f928f88ddc480dd31a0d3c75255"
server_key_iv: "147fde4a9e5933637d1be22e50319f97"
server_key_version: "encryptionkey@0"
I20260925 13:08:03.035712 10266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:08:03.052922 10266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:08:03.053184 10266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:08:03.053326 10266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:08:03.053535 10266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:08:03.053833 10266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:08:03.053887 10266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:03.053934 10266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:08:03.053960 10266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:03.059499 10266 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:44097
I20260925 13:08:03.059537 10393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:44097 every 8 connection(s)
I20260925 13:08:03.059857 10266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:08:03.064821 10134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:03.060611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:51361 (local address 127.0.206.126:43433)
0925 13:08:03.060691 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:03.060694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:03.060880 (+ 186us) server_negotiation.cc:407] Connection header received
0925 13:08:03.061320 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:03.061321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:03.061331 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:03.061385 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:03.062014 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:03.062492 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:03.063483 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:03.063673 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:03.063740 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:03.064579 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:08:03.064581 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:08:03.064581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:08:03.064585 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:08:03.064600 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:08:03.064604 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:08:03.064658 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:03.064737 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 13:08:03.064760 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 13:08:03.064972 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:03.060567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43433 (local address 127.0.206.66:51361)
0925 13:08:03.060770 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:03.060800 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:03.061231 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:03.061466 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:03.061469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:03.061477 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:03.061914 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:03.061918 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:03.062573 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:03.062573 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:03.063380 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:03.063383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:03.063746 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:03.063747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:03.063846 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:03.064388 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:08:03.064388 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:03.064443 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:08:03.064469 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:08:03.064668 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:08:03.064671 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:08:03.064674 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:08:03.064723 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 13:08:03.064796 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":98,"threads_started":1}
I20260925 13:08:03.065497 10394 heartbeater.cc:347] Connected to a master server at 127.0.206.126:43433
I20260925 13:08:03.065573 10394 heartbeater.cc:464] Registering TS with master...
I20260925 13:08:03.065771 10394 heartbeater.cc:511] Master 127.0.206.126:43433 requested a full tablet report, sending...
I20260925 13:08:03.066123 10075 ts_manager.cc:194] Registered new tserver with Master: 5beda895a5b24df0a09075131547ec1d (127.0.206.66:44097)
I20260925 13:08:03.066511 10075 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.66:51361
I20260925 13:08:03.066825 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 10266
I20260925 13:08:03.066921 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:08:03.067099 825 external_mini_cluster.cc:1428] Setting key 9d6225b8a5a2f7eeaaf71b8af9ed787f
I20260925 13:08:03.068002 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43433
--builtin_ntp_servers=127.0.206.84:34101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 13:08:03.145594 10397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:08:03.145784 10397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:08:03.145804 10397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:08:03.146981 10397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:08:03.147017 10397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:08:03.147042 10397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:08:03.148372 10397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:34101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:08:03.148597 10397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:08:03.148872 10397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:08:03.153658 10397 server_base.cc:1034] running on GCE node
W20260925 13:08:03.153596 10403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:03.153616 10405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:03.153592 10402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:03.154083 10397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:08:03.154327 10397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:08:03.155481 10397 hybrid_clock.cc:648] HybridClock initialized: now 1790341683155466 us; error 28 us; skew 500 ppm
I20260925 13:08:03.156772 10397 webserver.cc:460] Webserver started at http://127.0.206.67:42885/ using document root <none> and password file <none>
I20260925 13:08:03.157002 10397 fs_manager.cc:359] Metadata directory not provided
I20260925 13:08:03.157073 10397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:08:03.157204 10397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:08:03.158272 10397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "9fcd329f03b64e6ab8ccadb974459175"
format_stamp: "Formatted at 2026-09-25 13:08:03 on dist-test-slave-chkf"
server_key: "36925150205e2d056b57c3b505dc43ed"
server_key_iv: "67d70c6bf56edd0e1b0862eb7aaf9669"
server_key_version: "encryptionkey@0"
I20260925 13:08:03.158622 10397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "9fcd329f03b64e6ab8ccadb974459175"
format_stamp: "Formatted at 2026-09-25 13:08:03 on dist-test-slave-chkf"
server_key: "36925150205e2d056b57c3b505dc43ed"
server_key_iv: "67d70c6bf56edd0e1b0862eb7aaf9669"
server_key_version: "encryptionkey@0"
I20260925 13:08:03.159907 10397 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:08:03.160594 10411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:03.160791 10397 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:08:03.160939 10397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "9fcd329f03b64e6ab8ccadb974459175"
format_stamp: "Formatted at 2026-09-25 13:08:03 on dist-test-slave-chkf"
server_key: "36925150205e2d056b57c3b505dc43ed"
server_key_iv: "67d70c6bf56edd0e1b0862eb7aaf9669"
server_key_version: "encryptionkey@0"
I20260925 13:08:03.161046 10397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:08:03.170925 10397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:08:03.171170 10397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:08:03.171314 10397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:08:03.171514 10397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:08:03.171808 10397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:08:03.171859 10397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:03.171900 10397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:08:03.171926 10397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:03.177304 10397 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:46339
I20260925 13:08:03.177350 10524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:46339 every 8 connection(s)
I20260925 13:08:03.177627 10397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:08:03.182268 10134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:03.178283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:43533 (local address 127.0.206.126:43433)
0925 13:08:03.178342 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:03.178344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:03.178541 (+ 197us) server_negotiation.cc:407] Connection header received
0925 13:08:03.179008 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:03.179009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:03.179017 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:03.179076 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:03.179659 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:03.180129 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:03.181033 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:03.181205 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:03.181252 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:03.182008 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:08:03.182009 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:08:03.182009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:08:03.182013 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:08:03.182022 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:08:03.182025 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:08:03.182061 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:03.182145 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 13:08:03.182172 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 13:08:03.182288 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 10397
I20260925 13:08:03.182375 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:08:03.182401 10527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:03.178260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43433 (local address 127.0.206.67:43533)
0925 13:08:03.178428 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:03.178463 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:03.178912 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:03.179123 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:03.179126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:03.179133 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:03.179588 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:03.179592 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:03.180202 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:03.180202 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:03.180953 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:03.180955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:03.181270 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:03.181271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:03.181365 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:03.181890 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:08:03.181891 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:03.181915 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:08:03.181934 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:08:03.182063 (+ 129us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:08:03.182065 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:08:03.182067 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:08:03.182110 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 13:08:03.182176 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260925 13:08:03.182561 825 external_mini_cluster.cc:1428] Setting key 1cb87b7a0a74072f417de99f2ff669c7
I20260925 13:08:03.182914 10525 heartbeater.cc:347] Connected to a master server at 127.0.206.126:43433
I20260925 13:08:03.183001 10525 heartbeater.cc:464] Registering TS with master...
I20260925 13:08:03.183177 10525 heartbeater.cc:511] Master 127.0.206.126:43433 requested a full tablet report, sending...
I20260925 13:08:03.183525 10075 ts_manager.cc:194] Registered new tserver with Master: 9fcd329f03b64e6ab8ccadb974459175 (127.0.206.67:46339)
I20260925 13:08:03.183882 10075 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.67:43533
I20260925 13:08:03.194238 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 13:08:03.195932 10134 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:08:03.195550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44008 (local address 127.0.206.126:43433)
0925 13:08:03.195600 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:03.195602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:03.195674 (+ 72us) server_negotiation.cc:407] Connection header received
0925 13:08:03.195718 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:03.195719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:03.195727 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:03.195774 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:03.195879 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44008: BlockingRecv error: recv got EOF from 127.0.0.1:44008 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 13:08:03.195992 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 10135
I20260925 13:08:03.200834 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 10266
I20260925 13:08:03.205199 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 10397
I20260925 13:08:03.209085 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 10044
2026-09-25T13:08:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (602 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-25T13:08:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:08:03Z Disabled control of system clock
I20260925 13:08:03.234534 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:33881
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:42679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:33881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 13:08:03.312444 10536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:08:03.312628 10536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:08:03.312649 10536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:08:03.313694 10536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:08:03.313726 10536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:08:03.313738 10536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:08:03.313751 10536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:08:03.313763 10536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:08:03.315023 10536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:33881
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:33881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:08:03.315196 10536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:08:03.315418 10536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:08:03.319898 10544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:03.319898 10541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:03.319972 10542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:03.320242 10536 server_base.cc:1034] running on GCE node
I20260925 13:08:03.320480 10536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:08:03.320737 10536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:08:03.321892 10536 hybrid_clock.cc:648] HybridClock initialized: now 1790341683321862 us; error 41 us; skew 500 ppm
I20260925 13:08:03.323128 10536 webserver.cc:460] Webserver started at http://127.0.206.126:41487/ using document root <none> and password file <none>
I20260925 13:08:03.323341 10536 fs_manager.cc:359] Metadata directory not provided
I20260925 13:08:03.323386 10536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:08:03.323480 10536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:08:03.324514 10536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "05fce8b08564425eaf75e3908c7536f9"
format_stamp: "Formatted at 2026-09-25 13:08:03 on dist-test-slave-chkf"
server_key: "d4ae05bfa242e1d43c38052ecb2c2357"
server_key_iv: "e0f7177c023062db4924b6c1da177f13"
server_key_version: "encryptionkey@0"
I20260925 13:08:03.324834 10536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "05fce8b08564425eaf75e3908c7536f9"
format_stamp: "Formatted at 2026-09-25 13:08:03 on dist-test-slave-chkf"
server_key: "d4ae05bfa242e1d43c38052ecb2c2357"
server_key_iv: "e0f7177c023062db4924b6c1da177f13"
server_key_version: "encryptionkey@0"
I20260925 13:08:03.326056 10536 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:08:03.326737 10550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:03.326913 10536 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:08:03.327037 10536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "05fce8b08564425eaf75e3908c7536f9"
format_stamp: "Formatted at 2026-09-25 13:08:03 on dist-test-slave-chkf"
server_key: "d4ae05bfa242e1d43c38052ecb2c2357"
server_key_iv: "e0f7177c023062db4924b6c1da177f13"
server_key_version: "encryptionkey@0"
I20260925 13:08:03.327097 10536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:08:03.330575 10536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:08:03.330861 10536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:08:03.330972 10536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:08:03.334966 10536 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:33881
I20260925 13:08:03.334992 10602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:33881 every 8 connection(s)
I20260925 13:08:03.335327 10536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:08:03.335997 10603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:08:03.338393 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 10536
I20260925 13:08:03.338495 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:08:03.338644 10603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9: Bootstrap starting.
I20260925 13:08:03.338682 825 external_mini_cluster.cc:1428] Setting key fe842f958868cbfe16122f04e106097d
I20260925 13:08:03.339301 10603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9: Neither blocks nor log segments found. Creating new log.
I20260925 13:08:03.339582 10603 log.cc:824] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9: Log is configured to *not* fsync() on all Append() calls
I20260925 13:08:03.340240 10603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9: No bootstrap required, opened a new log
I20260925 13:08:03.341094 10606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:03.339161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57900 (local address 127.0.206.126:33881)
0925 13:08:03.339362 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:03.339372 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:03.339400 (+ 28us) server_negotiation.cc:407] Connection header received
0925 13:08:03.339834 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:03.339841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:03.339877 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:03.340015 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:03.340675 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:08:03.340679 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:08:03.340679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:08:03.340683 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:08:03.340724 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:08:03.340739 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:08:03.340838 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:03.340891 (+ 53us) server_negotiation.cc:299] Negotiation successful
0925 13:08:03.340944 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260925 13:08:03.341914 10603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05fce8b08564425eaf75e3908c7536f9" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33881 } }
I20260925 13:08:03.342031 10603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:08:03.342051 10603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05fce8b08564425eaf75e3908c7536f9, State: Initialized, Role: FOLLOWER
I20260925 13:08:03.342151 10603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05fce8b08564425eaf75e3908c7536f9" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33881 } }
I20260925 13:08:03.342226 10603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:08:03.342247 10603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:08:03.342273 10603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:08:03.343057 10603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05fce8b08564425eaf75e3908c7536f9" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33881 } }
I20260925 13:08:03.343140 10603 leader_election.cc:302] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05fce8b08564425eaf75e3908c7536f9; no voters:
I20260925 13:08:03.343269 10603 leader_election.cc:288] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:08:03.343298 10608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:08:03.343392 10608 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9 [term 1 LEADER]: Becoming Leader. State: Replica: 05fce8b08564425eaf75e3908c7536f9, State: Running, Role: LEADER
I20260925 13:08:03.343484 10608 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05fce8b08564425eaf75e3908c7536f9" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33881 } }
I20260925 13:08:03.343518 10603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:08:03.343792 10608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05fce8b08564425eaf75e3908c7536f9. Latest consensus state: current_term: 1 leader_uuid: "05fce8b08564425eaf75e3908c7536f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05fce8b08564425eaf75e3908c7536f9" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33881 } } }
I20260925 13:08:03.343870 10608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9 [sys.catalog]: This master's current role is: LEADER
I20260925 13:08:03.343801 10609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05fce8b08564425eaf75e3908c7536f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05fce8b08564425eaf75e3908c7536f9" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33881 } } }
I20260925 13:08:03.343920 10609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9 [sys.catalog]: This master's current role is: LEADER
I20260925 13:08:03.344597 10613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:08:03.345031 10613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:08:03.346499 10613 catalog_manager.cc:1347] Generated new cluster ID: f9a98466f28847c1b55423bd33966c8b
I20260925 13:08:03.346546 10613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:08:03.359862 10613 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:08:03.360010 10613 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:08:03.365962 10613 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05fce8b08564425eaf75e3908c7536f9: Generated new TSK 0
I20260925 13:08:03.366122 10613 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:08:03.372287 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33881
--builtin_ntp_servers=127.0.206.84:42679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 13:08:03.456651 10627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:08:03.456941 10627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:08:03.456977 10627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:08:03.458211 10627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:08:03.458268 10627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:08:03.458310 10627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:08:03.459712 10627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:08:03.459964 10627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:08:03.460245 10627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:08:03.465886 10632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:03.465977 10635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:03.466094 10627 server_base.cc:1034] running on GCE node
W20260925 13:08:03.465943 10633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:03.466398 10627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:08:03.466658 10627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:08:03.467823 10627 hybrid_clock.cc:648] HybridClock initialized: now 1790341683467799 us; error 44 us; skew 500 ppm
I20260925 13:08:03.469106 10627 webserver.cc:460] Webserver started at http://127.0.206.65:42127/ using document root <none> and password file <none>
I20260925 13:08:03.469338 10627 fs_manager.cc:359] Metadata directory not provided
I20260925 13:08:03.469408 10627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:08:03.469539 10627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:08:03.470525 10627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "9a97d2bc56fa4cc195aae419f16364e1"
format_stamp: "Formatted at 2026-09-25 13:08:03 on dist-test-slave-chkf"
server_key: "6d5848d294cdf1a51c594f4f22a493c7"
server_key_iv: "a9d54ebd3a18a65523475375b831d45c"
server_key_version: "encryptionkey@0"
I20260925 13:08:03.470897 10627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "9a97d2bc56fa4cc195aae419f16364e1"
format_stamp: "Formatted at 2026-09-25 13:08:03 on dist-test-slave-chkf"
server_key: "6d5848d294cdf1a51c594f4f22a493c7"
server_key_iv: "a9d54ebd3a18a65523475375b831d45c"
server_key_version: "encryptionkey@0"
I20260925 13:08:03.472244 10627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:08:03.472898 10641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:03.473069 10627 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:08:03.473161 10627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "9a97d2bc56fa4cc195aae419f16364e1"
format_stamp: "Formatted at 2026-09-25 13:08:03 on dist-test-slave-chkf"
server_key: "6d5848d294cdf1a51c594f4f22a493c7"
server_key_iv: "a9d54ebd3a18a65523475375b831d45c"
server_key_version: "encryptionkey@0"
I20260925 13:08:03.473244 10627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:08:03.476478 10627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:08:03.476722 10627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:08:03.476876 10627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:08:03.477082 10627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:08:03.477370 10627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:08:03.477421 10627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:03.477460 10627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:08:03.477486 10627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:03.482884 10627 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:42627
I20260925 13:08:03.482928 10754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:42627 every 8 connection(s)
I20260925 13:08:03.483196 10627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:08:03.485975 10606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:03.483789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:46579 (local address 127.0.206.126:33881)
0925 13:08:03.483850 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:03.483852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:03.484093 (+ 241us) server_negotiation.cc:407] Connection header received
0925 13:08:03.484535 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:03.484536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:03.484541 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:03.484588 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:03.485712 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:08:03.485713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:08:03.485714 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:08:03.485718 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:08:03.485733 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:08:03.485737 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:08:03.485789 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:03.485892 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 13:08:03.485906 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 13:08:03.486162 10757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:03.483797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33881 (local address 127.0.206.65:46579)
0925 13:08:03.483990 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:03.484018 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:03.484438 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:03.484654 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:03.484657 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:03.484665 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:03.485567 (+ 902us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:08:03.485567 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:03.485592 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:08:03.485616 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:08:03.485786 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:08:03.485790 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:08:03.485793 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:08:03.485899 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 13:08:03.485956 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":98,"threads_started":1}
I20260925 13:08:03.486621 10755 heartbeater.cc:347] Connected to a master server at 127.0.206.126:33881
I20260925 13:08:03.486737 10755 heartbeater.cc:464] Registering TS with master...
I20260925 13:08:03.486876 10755 heartbeater.cc:511] Master 127.0.206.126:33881 requested a full tablet report, sending...
I20260925 13:08:03.487283 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 10627
I20260925 13:08:03.487382 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:08:03.487389 10567 ts_manager.cc:194] Registered new tserver with Master: 9a97d2bc56fa4cc195aae419f16364e1 (127.0.206.65:42627)
I20260925 13:08:03.487572 825 external_mini_cluster.cc:1428] Setting key 477262f8bee7db8f36736565088eb9ed
I20260925 13:08:03.488389 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33881
--builtin_ntp_servers=127.0.206.84:42679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 13:08:03.567016 10758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:08:03.567194 10758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:08:03.567215 10758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:08:03.568362 10758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:08:03.568403 10758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:08:03.568430 10758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:08:03.569789 10758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:08:03.569974 10758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:08:03.570196 10758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:08:03.575305 10763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:03.575291 10764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:03.575400 10758 server_base.cc:1034] running on GCE node
W20260925 13:08:03.575294 10766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:03.575786 10758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:08:03.576036 10758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:08:03.577205 10758 hybrid_clock.cc:648] HybridClock initialized: now 1790341683577158 us; error 65 us; skew 500 ppm
I20260925 13:08:03.578456 10758 webserver.cc:460] Webserver started at http://127.0.206.66:43697/ using document root <none> and password file <none>
I20260925 13:08:03.578698 10758 fs_manager.cc:359] Metadata directory not provided
I20260925 13:08:03.578800 10758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:08:03.578920 10758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:08:03.579913 10758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "1c79da9e8bff4323ae88b84c606fa3c0"
format_stamp: "Formatted at 2026-09-25 13:08:03 on dist-test-slave-chkf"
server_key: "25f305f33fc4dca7e4ac881f97efe4c4"
server_key_iv: "59ebe7f2ba5884f41e689c46caaa127b"
server_key_version: "encryptionkey@0"
I20260925 13:08:03.580253 10758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "1c79da9e8bff4323ae88b84c606fa3c0"
format_stamp: "Formatted at 2026-09-25 13:08:03 on dist-test-slave-chkf"
server_key: "25f305f33fc4dca7e4ac881f97efe4c4"
server_key_iv: "59ebe7f2ba5884f41e689c46caaa127b"
server_key_version: "encryptionkey@0"
I20260925 13:08:03.581502 10758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:08:03.582175 10772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:03.582334 10758 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:08:03.582428 10758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "1c79da9e8bff4323ae88b84c606fa3c0"
format_stamp: "Formatted at 2026-09-25 13:08:03 on dist-test-slave-chkf"
server_key: "25f305f33fc4dca7e4ac881f97efe4c4"
server_key_iv: "59ebe7f2ba5884f41e689c46caaa127b"
server_key_version: "encryptionkey@0"
I20260925 13:08:03.582505 10758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:08:03.585630 10758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:08:03.585901 10758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:08:03.586030 10758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:08:03.586234 10758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:08:03.586513 10758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:08:03.586565 10758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:03.586601 10758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:08:03.586628 10758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:03.591910 10758 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:38153
I20260925 13:08:03.591958 10885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:38153 every 8 connection(s)
I20260925 13:08:03.592229 10758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:08:03.592883 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 10758
I20260925 13:08:03.592990 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:08:03.593183 825 external_mini_cluster.cc:1428] Setting key 0fd92fd915eef68dce86a235bdc5ceee
I20260925 13:08:03.594102 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33881
--builtin_ntp_servers=127.0.206.84:42679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260925 13:08:03.595314 10606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:03.592854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:43495 (local address 127.0.206.126:33881)
0925 13:08:03.592900 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:03.592902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:03.593128 (+ 226us) server_negotiation.cc:407] Connection header received
0925 13:08:03.593683 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:03.593684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:03.593688 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:03.593800 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:03.595039 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:08:03.595041 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:08:03.595041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:08:03.595046 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:08:03.595059 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:08:03.595063 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:08:03.595120 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:03.595204 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 13:08:03.595220 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 13:08:03.595705 10888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:03.592864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33881 (local address 127.0.206.66:43495)
0925 13:08:03.593024 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:03.593060 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:03.593596 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:03.593865 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:03.593869 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:03.593880 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:03.594896 (+ 1016us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:08:03.594897 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:03.594923 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:08:03.594948 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:08:03.595120 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:08:03.595125 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:08:03.595128 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:08:03.595420 (+ 292us) client_negotiation.cc:240] Negotiation successful
0925 13:08:03.595483 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
I20260925 13:08:03.596164 10886 heartbeater.cc:347] Connected to a master server at 127.0.206.126:33881
I20260925 13:08:03.596249 10886 heartbeater.cc:464] Registering TS with master...
I20260925 13:08:03.596403 10886 heartbeater.cc:511] Master 127.0.206.126:33881 requested a full tablet report, sending...
I20260925 13:08:03.596711 10567 ts_manager.cc:194] Registered new tserver with Master: 1c79da9e8bff4323ae88b84c606fa3c0 (127.0.206.66:38153)
W20260925 13:08:03.672683 10889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:08:03.672892 10889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:08:03.672928 10889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:08:03.674062 10889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:08:03.674120 10889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:08:03.674163 10889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:08:03.675521 10889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:08:03.675763 10889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:08:03.676044 10889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:08:03.680966 10897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:03.680992 10895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:03.680989 10894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:03.681219 10889 server_base.cc:1034] running on GCE node
I20260925 13:08:03.681454 10889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:08:03.681730 10889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:08:03.682888 10889 hybrid_clock.cc:648] HybridClock initialized: now 1790341683682864 us; error 39 us; skew 500 ppm
I20260925 13:08:03.684167 10889 webserver.cc:460] Webserver started at http://127.0.206.67:42091/ using document root <none> and password file <none>
I20260925 13:08:03.684401 10889 fs_manager.cc:359] Metadata directory not provided
I20260925 13:08:03.684473 10889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:08:03.684624 10889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:08:03.685653 10889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "3e142a668f9444059d8a6d8d2769739e"
format_stamp: "Formatted at 2026-09-25 13:08:03 on dist-test-slave-chkf"
server_key: "418b4d2b25fd0ce9263e953288433b9d"
server_key_iv: "0407d56541e6d4c6b1a0429ffb3f443f"
server_key_version: "encryptionkey@0"
I20260925 13:08:03.686033 10889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "3e142a668f9444059d8a6d8d2769739e"
format_stamp: "Formatted at 2026-09-25 13:08:03 on dist-test-slave-chkf"
server_key: "418b4d2b25fd0ce9263e953288433b9d"
server_key_iv: "0407d56541e6d4c6b1a0429ffb3f443f"
server_key_version: "encryptionkey@0"
I20260925 13:08:03.687368 10889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:08:03.688062 10903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:03.688225 10889 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:08:03.688319 10889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "3e142a668f9444059d8a6d8d2769739e"
format_stamp: "Formatted at 2026-09-25 13:08:03 on dist-test-slave-chkf"
server_key: "418b4d2b25fd0ce9263e953288433b9d"
server_key_iv: "0407d56541e6d4c6b1a0429ffb3f443f"
server_key_version: "encryptionkey@0"
I20260925 13:08:03.688398 10889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:08:03.691591 10889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:08:03.691831 10889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:08:03.691967 10889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:08:03.692162 10889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:08:03.692447 10889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:08:03.692499 10889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:03.692536 10889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:08:03.692564 10889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:03.698004 10889 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:38761
I20260925 13:08:03.698048 11016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:38761 every 8 connection(s)
I20260925 13:08:03.698323 10889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:08:03.698753 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 10889
I20260925 13:08:03.698859 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:08:03.699069 825 external_mini_cluster.cc:1428] Setting key 6ba167010fd726c30c14bf18a26911b7
I20260925 13:08:03.701316 10606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:03.698944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:37233 (local address 127.0.206.126:33881)
0925 13:08:03.699006 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:03.699009 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:03.699277 (+ 268us) server_negotiation.cc:407] Connection header received
0925 13:08:03.699885 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:03.699886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:03.699892 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:03.699937 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:03.701085 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:08:03.701086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:08:03.701087 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:08:03.701092 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:08:03.701106 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:08:03.701110 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:08:03.701166 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:03.701236 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 13:08:03.701249 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 13:08:03.701493 11019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:03.698957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33881 (local address 127.0.206.67:37233)
0925 13:08:03.699140 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:03.699182 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:03.699789 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:03.699998 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:03.700002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:03.700010 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:03.700935 (+ 925us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:08:03.700935 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:03.700972 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:08:03.700997 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:08:03.701166 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:08:03.701171 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:08:03.701173 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:08:03.701248 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 13:08:03.701302 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260925 13:08:03.701889 11017 heartbeater.cc:347] Connected to a master server at 127.0.206.126:33881
I20260925 13:08:03.701977 11017 heartbeater.cc:464] Registering TS with master...
I20260925 13:08:03.702075 11017 heartbeater.cc:511] Master 127.0.206.126:33881 requested a full tablet report, sending...
I20260925 13:08:03.702382 10567 ts_manager.cc:194] Registered new tserver with Master: 3e142a668f9444059d8a6d8d2769739e (127.0.206.67:38761)
I20260925 13:08:03.710629 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 13:08:03.712414 10606 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:08:03.711934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57910 (local address 127.0.206.126:33881)
0925 13:08:03.712022 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:03.712024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:03.712125 (+ 101us) server_negotiation.cc:407] Connection header received
0925 13:08:03.712189 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:03.712190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:03.712195 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:03.712244 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:03.712349 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57910: BlockingRecv error: recv got EOF from 127.0.0.1:57910 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 13:08:03.712509 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 10627
I20260925 13:08:03.716714 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 10758
I20260925 13:08:03.721024 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 10889
I20260925 13:08:03.724902 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 10536
2026-09-25T13:08:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (516 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:08:03 dist-test-slave-chkf krb5kdc[11027](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:08:03 dist-test-slave-chkf krb5kdc[11027](info): set up 2 sockets
Sep 25 13:08:03 dist-test-slave-chkf krb5kdc[11027](info): commencing operation
krb5kdc: starting...
W20260925 13:08:04.755544 825 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:08:04 dist-test-slave-chkf krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:08:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:08:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:08:04.869067 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:39503
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:39503
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:08:04.945227 11040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:08:04.945444 11040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:08:04.945482 11040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:08:04.946590 11040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:08:04.946655 11040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:08:04.946679 11040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:08:04.946705 11040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:08:04.946771 11040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:08:04.948215 11040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:39503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:39503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:08:04.948482 11040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:08:04.948693 11040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:08:04.953359 11040 server_base.cc:1034] running on GCE node
W20260925 13:08:04.953372 11045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:04.953344 11046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:04.953336 11048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:04.953819 11040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:08:04.954066 11040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:08:04.955205 11040 hybrid_clock.cc:648] HybridClock initialized: now 1790341684955192 us; error 28 us; skew 500 ppm
Sep 25 13:08:04 dist-test-slave-chkf krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341684, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:08:04.957508 11040 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:08:04.957914 11040 webserver.cc:460] Webserver started at http://127.0.206.126:34569/ using document root <none> and password file <none>
I20260925 13:08:04.958143 11040 fs_manager.cc:359] Metadata directory not provided
I20260925 13:08:04.958187 11040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:08:04.958333 11040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:08:04.959398 11040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "8f090d52c84a4b51bde0cde0834da743"
format_stamp: "Formatted at 2026-09-25 13:08:04 on dist-test-slave-chkf"
server_key: "647aa5f105ee7f5cf17fed7b3d7632e9"
server_key_iv: "8223317feca2193c0cb8c9014aefc0b9"
server_key_version: "encryptionkey@0"
I20260925 13:08:04.959744 11040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "8f090d52c84a4b51bde0cde0834da743"
format_stamp: "Formatted at 2026-09-25 13:08:04 on dist-test-slave-chkf"
server_key: "647aa5f105ee7f5cf17fed7b3d7632e9"
server_key_iv: "8223317feca2193c0cb8c9014aefc0b9"
server_key_version: "encryptionkey@0"
I20260925 13:08:04.960978 11040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:08:04.961705 11055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:04.961862 11040 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:08:04.961961 11040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "8f090d52c84a4b51bde0cde0834da743"
format_stamp: "Formatted at 2026-09-25 13:08:04 on dist-test-slave-chkf"
server_key: "647aa5f105ee7f5cf17fed7b3d7632e9"
server_key_iv: "8223317feca2193c0cb8c9014aefc0b9"
server_key_version: "encryptionkey@0"
I20260925 13:08:04.962044 11040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:08:04.973383 11040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:08:04.975109 11040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:08:04.975260 11040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:08:04.978824 11040 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:39503
I20260925 13:08:04.978875 11107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:39503 every 8 connection(s)
I20260925 13:08:04.979207 11040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:08:04.980065 11108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:08:04.982061 11108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743: Bootstrap starting.
I20260925 13:08:04.982599 11108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743: Neither blocks nor log segments found. Creating new log.
I20260925 13:08:04.982928 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 11040
I20260925 13:08:04.982952 11108 log.cc:824] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743: Log is configured to *not* fsync() on all Append() calls
I20260925 13:08:04.983032 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:08:04.983268 825 external_mini_cluster.cc:1428] Setting key 4e508fdb2fc45576db55c751175c18c3
I20260925 13:08:04.983767 11108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743: No bootstrap required, opened a new log
I20260925 13:08:04.985698 11108 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f090d52c84a4b51bde0cde0834da743" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39503 } }
I20260925 13:08:04.985863 11108 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:08:04.985908 11108 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f090d52c84a4b51bde0cde0834da743, State: Initialized, Role: FOLLOWER
I20260925 13:08:04.986049 11108 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f090d52c84a4b51bde0cde0834da743" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39503 } }
I20260925 13:08:04.986121 11108 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:08:04.986151 11108 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:08:04.986195 11108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:08:04.986845 11108 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f090d52c84a4b51bde0cde0834da743" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39503 } }
I20260925 13:08:04.986946 11108 leader_election.cc:302] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f090d52c84a4b51bde0cde0834da743; no voters:
I20260925 13:08:04.987077 11108 leader_election.cc:288] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:08:04.987133 11113 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:08:04.987268 11108 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:08:04.987278 11113 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743 [term 1 LEADER]: Becoming Leader. State: Replica: 8f090d52c84a4b51bde0cde0834da743, State: Running, Role: LEADER
I20260925 13:08:04.988655 11113 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f090d52c84a4b51bde0cde0834da743" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39503 } }
I20260925 13:08:04.988982 11113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f090d52c84a4b51bde0cde0834da743. Latest consensus state: current_term: 1 leader_uuid: "8f090d52c84a4b51bde0cde0834da743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f090d52c84a4b51bde0cde0834da743" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39503 } } }
I20260925 13:08:04.989111 11113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743 [sys.catalog]: This master's current role is: LEADER
I20260925 13:08:04.989388 11115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f090d52c84a4b51bde0cde0834da743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f090d52c84a4b51bde0cde0834da743" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39503 } } }
I20260925 13:08:04.989464 11115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743 [sys.catalog]: This master's current role is: LEADER
I20260925 13:08:04.989621 11127 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:08:04.990139 11127 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 13:08:04 dist-test-slave-chkf krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:08:04.991653 11127 catalog_manager.cc:1347] Generated new cluster ID: 7f9839f213c5437e9640e1e5ea2f6b34
I20260925 13:08:04.991703 11127 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:08:04.995241 11112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:04.983736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47850 (local address 127.0.206.126:39503)
0925 13:08:04.983954 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:04.983962 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:04.983994 (+ 32us) server_negotiation.cc:407] Connection header received
0925 13:08:04.984926 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:04.984934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:04.984992 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:04.985131 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:04.985670 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:04.987303 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:04.988237 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:04.988844 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:04.988958 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:04.991725 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:08:04.991728 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:08:04.991729 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:08:04.991740 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:08:04.993658 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:04.994042 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:04.994045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:04.994046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:04.994148 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:04.994349 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:04.994350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:04.994351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:04.994636 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:08:04.994813 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:04.994894 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 13:08:04.995024 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":80,"threads_started":1}
I20260925 13:08:05.029489 11127 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:08:05.030045 11127 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:08:05.034480 11127 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743: Generated new TSK 0
I20260925 13:08:05.034651 11127 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260925 13:08:05.061744 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39503
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:08:05.141201 11134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:08:05.141390 11134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:08:05.141413 11134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:08:05.142547 11134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:08:05.142588 11134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:08:05.142639 11134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:08:05.144083 11134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:08:05.144304 11134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:08:05.144486 11134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:08:05.149219 11140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:05.149219 11139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:05.149224 11142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:05.149245 11134 server_base.cc:1034] running on GCE node
I20260925 13:08:05.149605 11134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:08:05.149845 11134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:08:05.151010 11134 hybrid_clock.cc:648] HybridClock initialized: now 1790341685151000 us; error 26 us; skew 500 ppm
Sep 25 13:08:05 dist-test-slave-chkf krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341685, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:08:05.153250 11134 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:08:05.153656 11134 webserver.cc:460] Webserver started at http://127.0.206.65:41575/ using document root <none> and password file <none>
I20260925 13:08:05.153889 11134 fs_manager.cc:359] Metadata directory not provided
I20260925 13:08:05.153962 11134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:08:05.154119 11134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:08:05.155202 11134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "aa78d57eef9d4be5aff427dc29d9ea18"
format_stamp: "Formatted at 2026-09-25 13:08:05 on dist-test-slave-chkf"
server_key: "c342cab70787aed49da78d90a7158ca5"
server_key_iv: "37ff1d5ee0d8a467caf0ac1bf684023f"
server_key_version: "encryptionkey@0"
I20260925 13:08:05.155543 11134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "aa78d57eef9d4be5aff427dc29d9ea18"
format_stamp: "Formatted at 2026-09-25 13:08:05 on dist-test-slave-chkf"
server_key: "c342cab70787aed49da78d90a7158ca5"
server_key_iv: "37ff1d5ee0d8a467caf0ac1bf684023f"
server_key_version: "encryptionkey@0"
I20260925 13:08:05.156878 11134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:08:05.157599 11149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:05.157765 11134 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:08:05.157877 11134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "aa78d57eef9d4be5aff427dc29d9ea18"
format_stamp: "Formatted at 2026-09-25 13:08:05 on dist-test-slave-chkf"
server_key: "c342cab70787aed49da78d90a7158ca5"
server_key_iv: "37ff1d5ee0d8a467caf0ac1bf684023f"
server_key_version: "encryptionkey@0"
I20260925 13:08:05.157948 11134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:08:05.169117 11134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:08:05.170990 11134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:08:05.171132 11134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:08:05.171339 11134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:08:05.171612 11134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:08:05.171663 11134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:05.171701 11134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:08:05.171729 11134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:05.177309 11134 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:43123
I20260925 13:08:05.177387 11262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:43123 every 8 connection(s)
I20260925 13:08:05.177661 11134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
Sep 25 13:08:05 dist-test-slave-chkf krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341685, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:08:05.185930 11112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.178363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:56001 (local address 127.0.206.126:39503)
0925 13:08:05.178420 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:05.178422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:05.178606 (+ 184us) server_negotiation.cc:407] Connection header received
0925 13:08:05.179110 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:05.179112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:05.179121 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:05.179171 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:05.180010 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.180499 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.181405 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.181584 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.181632 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:05.183669 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:08:05.183671 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:08:05.183671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:08:05.183677 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:08:05.184853 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:05.185188 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:05.185188 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:05.185189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:05.185234 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:05.185434 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:05.185435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:05.185435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:05.185526 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:08:05.185573 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:05.185806 (+ 233us) server_negotiation.cc:299] Negotiation successful
0925 13:08:05.185867 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 13:08:05.186110 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.178314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:39503 (local address 127.0.206.65:56001)
0925 13:08:05.178477 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:05.178515 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:05.179024 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:05.179227 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:05.179230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:05.179748 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:05.179935 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:05.179939 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.180573 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.180574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:05.181333 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:05.181335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.181651 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.181652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:05.181750 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:05.182257 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:08:05.182265 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:05.183588 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:08:05.184918 (+ 1330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:08:05.184924 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:08:05.184926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:08:05.185127 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:08:05.185287 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:08:05.185287 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:08:05.185288 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:08:05.185383 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:08:05.185577 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:08:05.185580 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:08:05.185679 (+ 99us) client_negotiation.cc:769] Sending connection context
0925 13:08:05.185783 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 13:08:05.185892 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260925 13:08:05.186435 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 11134
I20260925 13:08:05.186534 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:08:05.186595 11263 heartbeater.cc:347] Connected to a master server at 127.0.206.126:39503
I20260925 13:08:05.186702 11263 heartbeater.cc:464] Registering TS with master...
I20260925 13:08:05.186760 825 external_mini_cluster.cc:1428] Setting key e968e09d2dad84feb78da7ba8d3fa68f
I20260925 13:08:05.186920 11263 heartbeater.cc:511] Master 127.0.206.126:39503 requested a full tablet report, sending...
I20260925 13:08:05.187445 11071 ts_manager.cc:194] Registered new tserver with Master: aa78d57eef9d4be5aff427dc29d9ea18 (127.0.206.65:43123)
I20260925 13:08:05.188133 11071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:56001
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260925 13:08:05.211575 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39503
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:08:05.290781 11268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:08:05.290987 11268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:08:05.291023 11268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:08:05.292124 11268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:08:05.292188 11268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:08:05.292230 11268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:08:05.293605 11268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:08:05.293861 11268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:08:05.294085 11268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:08:05.299348 11276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:05.299340 11273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:05.299341 11274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:05.299422 11268 server_base.cc:1034] running on GCE node
I20260925 13:08:05.299787 11268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:08:05.300040 11268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:08:05.301199 11268 hybrid_clock.cc:648] HybridClock initialized: now 1790341685301196 us; error 36 us; skew 500 ppm
Sep 25 13:08:05 dist-test-slave-chkf krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341685, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:08:05.303563 11268 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:08:05.303954 11268 webserver.cc:460] Webserver started at http://127.0.206.66:39739/ using document root <none> and password file <none>
I20260925 13:08:05.304186 11268 fs_manager.cc:359] Metadata directory not provided
I20260925 13:08:05.304251 11268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:08:05.304396 11268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:08:05.305362 11268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "79d3562a70b04881835ece829a4fad5e"
format_stamp: "Formatted at 2026-09-25 13:08:05 on dist-test-slave-chkf"
server_key: "feeedfee4a2f88068aaa686eb7d52faa"
server_key_iv: "04afa8705ae7db35e343d8beb44890d1"
server_key_version: "encryptionkey@0"
I20260925 13:08:05.305697 11268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "79d3562a70b04881835ece829a4fad5e"
format_stamp: "Formatted at 2026-09-25 13:08:05 on dist-test-slave-chkf"
server_key: "feeedfee4a2f88068aaa686eb7d52faa"
server_key_iv: "04afa8705ae7db35e343d8beb44890d1"
server_key_version: "encryptionkey@0"
I20260925 13:08:05.306973 11268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:08:05.307657 11283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:05.307837 11268 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:08:05.307929 11268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "79d3562a70b04881835ece829a4fad5e"
format_stamp: "Formatted at 2026-09-25 13:08:05 on dist-test-slave-chkf"
server_key: "feeedfee4a2f88068aaa686eb7d52faa"
server_key_iv: "04afa8705ae7db35e343d8beb44890d1"
server_key_version: "encryptionkey@0"
I20260925 13:08:05.308007 11268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:08:05.319777 11268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:08:05.321396 11268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:08:05.321549 11268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:08:05.321746 11268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:08:05.322027 11268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:08:05.322080 11268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:05.322118 11268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:08:05.322146 11268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:05.327903 11268 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:42055
I20260925 13:08:05.327943 11396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:42055 every 8 connection(s)
I20260925 13:08:05.328238 11268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
Sep 25 13:08:05 dist-test-slave-chkf krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341685, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:08:05.335635 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 11268
I20260925 13:08:05.335732 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:08:05.335945 825 external_mini_cluster.cc:1428] Setting key d4c4f5c46005a22ca08042449dff0580
I20260925 13:08:05.336915 11112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.328860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:56637 (local address 127.0.206.126:39503)
0925 13:08:05.328926 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:05.328929 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:05.329139 (+ 210us) server_negotiation.cc:407] Connection header received
0925 13:08:05.329551 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:05.329552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:05.329561 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:05.329615 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:05.330431 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.331001 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.331913 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.332098 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.332157 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:05.334228 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:08:05.334229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:08:05.334229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:08:05.334235 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:08:05.335463 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:05.335942 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:05.335943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:05.335943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:05.335992 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:05.336249 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:05.336250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:05.336250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:05.336359 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:08:05.336433 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:05.336720 (+ 287us) server_negotiation.cc:299] Negotiation successful
0925 13:08:05.336805 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 13:08:05.336989 11399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.328842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:39503 (local address 127.0.206.66:56637)
0925 13:08:05.329030 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:05.329061 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:05.329471 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:05.329662 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:05.329665 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:05.330187 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:05.330349 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:05.330354 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.331074 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.331075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:05.331821 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:05.331823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.332171 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.332172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:05.332262 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:05.332745 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:08:05.332753 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:05.334145 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:08:05.335534 (+ 1389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:08:05.335540 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:08:05.335543 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:08:05.335852 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:08:05.336062 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:08:05.336063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:08:05.336063 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:08:05.336193 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:08:05.336424 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:08:05.336430 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:08:05.336560 (+ 130us) client_negotiation.cc:769] Sending connection context
0925 13:08:05.336677 (+ 117us) client_negotiation.cc:240] Negotiation successful
0925 13:08:05.336826 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":109,"threads_started":1}
I20260925 13:08:05.338297 11397 heartbeater.cc:347] Connected to a master server at 127.0.206.126:39503
I20260925 13:08:05.338384 11397 heartbeater.cc:464] Registering TS with master...
I20260925 13:08:05.338572 11397 heartbeater.cc:511] Master 127.0.206.126:39503 requested a full tablet report, sending...
I20260925 13:08:05.338948 11071 ts_manager.cc:194] Registered new tserver with Master: 79d3562a70b04881835ece829a4fad5e (127.0.206.66:42055)
I20260925 13:08:05.339457 11071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:56637
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260925 13:08:05.361574 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39503
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:08:05.444845 11402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:08:05.445132 11402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:08:05.445171 11402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:08:05.447000 11402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:08:05.447062 11402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:08:05.447106 11402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:08:05.448518 11402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:08:05.448774 11402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:08:05.449021 11402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:08:05.453809 11408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:05.453814 11410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:05.453855 11407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:05.454137 11402 server_base.cc:1034] running on GCE node
I20260925 13:08:05.454330 11402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:08:05.454561 11402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:08:05.455719 11402 hybrid_clock.cc:648] HybridClock initialized: now 1790341685455704 us; error 30 us; skew 500 ppm
Sep 25 13:08:05 dist-test-slave-chkf krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341685, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:08:05.458089 11402 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:08:05.458492 11402 webserver.cc:460] Webserver started at http://127.0.206.67:42239/ using document root <none> and password file <none>
I20260925 13:08:05.458763 11402 fs_manager.cc:359] Metadata directory not provided
I20260925 13:08:05.458859 11402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:08:05.459008 11402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:08:05.460016 11402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "bb1d2b3a4f554104a4b5869b50f13ede"
format_stamp: "Formatted at 2026-09-25 13:08:05 on dist-test-slave-chkf"
server_key: "da9d599953406fe8a28e26e8d9c079ba"
server_key_iv: "0bb5040ea7779d969addcad57bd064bb"
server_key_version: "encryptionkey@0"
I20260925 13:08:05.460350 11402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "bb1d2b3a4f554104a4b5869b50f13ede"
format_stamp: "Formatted at 2026-09-25 13:08:05 on dist-test-slave-chkf"
server_key: "da9d599953406fe8a28e26e8d9c079ba"
server_key_iv: "0bb5040ea7779d969addcad57bd064bb"
server_key_version: "encryptionkey@0"
I20260925 13:08:05.462077 11402 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260925 13:08:05.462950 11417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:05.463133 11402 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:08:05.463255 11402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "bb1d2b3a4f554104a4b5869b50f13ede"
format_stamp: "Formatted at 2026-09-25 13:08:05 on dist-test-slave-chkf"
server_key: "da9d599953406fe8a28e26e8d9c079ba"
server_key_iv: "0bb5040ea7779d969addcad57bd064bb"
server_key_version: "encryptionkey@0"
I20260925 13:08:05.463346 11402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:08:05.481611 11402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:08:05.483446 11402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:08:05.483604 11402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:08:05.483829 11402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:08:05.484122 11402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:08:05.484174 11402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:05.484215 11402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:08:05.484242 11402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:05.490170 11402 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:43429
I20260925 13:08:05.490219 11530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:43429 every 8 connection(s)
I20260925 13:08:05.490501 11402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:08:05.495844 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 11402
I20260925 13:08:05.495947 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:08:05.496137 825 external_mini_cluster.cc:1428] Setting key f0b773b3796a45c288a40cc2f3ea5390
Sep 25 13:08:05 dist-test-slave-chkf krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341685, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:08:05.499251 11112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.491118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:48691 (local address 127.0.206.126:39503)
0925 13:08:05.491160 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:05.491163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:05.491339 (+ 176us) server_negotiation.cc:407] Connection header received
0925 13:08:05.491850 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:05.491851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:05.491863 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:05.491931 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:05.492818 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.493321 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.494315 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.494526 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.494609 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:05.496989 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:08:05.496991 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:08:05.496991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:08:05.496999 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:08:05.498122 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:05.498474 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:05.498475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:05.498475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:05.498523 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:05.498766 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:05.498767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:05.498767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:05.498875 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:08:05.498933 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:05.499135 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 13:08:05.499190 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:08:05.499485 11533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.491086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:39503 (local address 127.0.206.67:48691)
0925 13:08:05.491213 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:05.491270 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:05.491742 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:05.491999 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:05.492003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:05.492545 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:05.492724 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:05.492729 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.493428 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.493429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:05.494212 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:05.494214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.494620 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.494621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:05.494801 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:05.495242 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:08:05.495253 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:05.496925 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:08:05.498187 (+ 1262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:08:05.498193 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:08:05.498194 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:08:05.498411 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:08:05.498579 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:08:05.498579 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:08:05.498579 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:08:05.498668 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:08:05.498928 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:08:05.498933 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:08:05.499037 (+ 104us) client_negotiation.cc:769] Sending connection context
0925 13:08:05.499122 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 13:08:05.499231 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":61,"threads_started":1}
I20260925 13:08:05.499918 11531 heartbeater.cc:347] Connected to a master server at 127.0.206.126:39503
I20260925 13:08:05.500015 11531 heartbeater.cc:464] Registering TS with master...
I20260925 13:08:05.500176 11531 heartbeater.cc:511] Master 127.0.206.126:39503 requested a full tablet report, sending...
I20260925 13:08:05.500537 11071 ts_manager.cc:194] Registered new tserver with Master: bb1d2b3a4f554104a4b5869b50f13ede (127.0.206.67:43429)
I20260925 13:08:05.500885 11071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:48691
I20260925 13:08:05.507874 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:08:05.514740 11112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.509127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47852 (local address 127.0.206.126:39503)
0925 13:08:05.509200 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:05.509202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:05.509292 (+ 90us) server_negotiation.cc:407] Connection header received
0925 13:08:05.509382 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:05.509383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:05.509393 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:05.509447 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:05.509977 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.510441 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.511266 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.511441 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.511491 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:05.512569 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:08:05.512570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:08:05.512570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:08:05.512576 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:08:05.513784 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:05.514102 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:05.514103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:05.514103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:05.514147 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:05.514279 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:05.514280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:05.514280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:05.514400 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:08:05.514472 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:05.514572 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 13:08:05.514627 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 13:08:05.516110 11071 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:08:05.517584 11071 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:08:05.523942 11541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.519763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41016 (local address 127.0.206.67:43429)
0925 13:08:05.520169 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:05.520173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:05.520182 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:08:05.520215 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:05.520216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:05.520229 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:05.520305 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:08:05.520484 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.521594 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.523023 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.523518 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.523604 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:05.523663 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:05.523826 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 13:08:05.523862 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":80,"threads_started":1}
I20260925 13:08:05.523939 11539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.519653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:43429 (local address 127.0.0.1:41016)
0925 13:08:05.519759 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:05.519795 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:05.519891 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:05.520322 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:05.520325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:05.520327 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:08:05.520395 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:05.520396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.521698 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.521698 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:05.522941 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:05.522943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.523616 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.523617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:05.523696 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:05.523709 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 13:08:05.523809 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 13:08:05.523872 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":39,"thread_start_us":65,"threads_started":1}
I20260925 13:08:05.524295 11540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.519856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:43123 (local address 127.0.0.1:45412)
0925 13:08:05.520105 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:05.520114 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:05.520159 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:05.520656 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:05.520657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:05.520659 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:08:05.520698 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:05.520700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.521568 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.521569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:05.522960 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:05.522962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.523908 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.523909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:05.523966 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:05.523971 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 13:08:05.524182 (+ 211us) client_negotiation.cc:240] Negotiation successful
0925 13:08:05.524223 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":29,"thread_start_us":47,"threads_started":1}
I20260925 13:08:05.524591 11544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.519972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45412 (local address 127.0.206.65:43123)
0925 13:08:05.520437 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:05.520441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:05.520451 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:08:05.520488 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:05.520490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:05.520505 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:05.520577 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:08:05.520790 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.521456 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.523056 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.523761 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.523821 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:05.523876 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:05.524474 (+ 598us) server_negotiation.cc:299] Negotiation successful
0925 13:08:05.524515 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":89,"threads_started":1}
I20260925 13:08:05.524791 11465 tablet_service.cc:1467] Processing CreateTablet for tablet 3d364670f38d4e4b948c7e44d1a0843b (DEFAULT_TABLE table=test-table [id=f0271c385774405f9d8460a82e26d17e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:08:05.524974 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.519661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:42055 (local address 127.0.0.1:57978)
0925 13:08:05.519956 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:05.519967 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:05.520018 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:05.520322 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:05.520325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:05.520330 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:08:05.520398 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:05.520399 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.522680 (+ 2281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.522681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:05.523729 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:05.523731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.524741 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.524742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:05.524807 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:05.524811 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:08:05.524873 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 13:08:05.524912 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":53,"threads_started":1}
I20260925 13:08:05.525169 11465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d364670f38d4e4b948c7e44d1a0843b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:08:05.525192 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.519950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57978 (local address 127.0.206.66:42055)
0925 13:08:05.520133 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:05.520138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:05.520150 (+ 12us) server_negotiation.cc:407] Connection header received
0925 13:08:05.520194 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:05.520196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:05.520213 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:05.520301 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:08:05.520485 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.521594 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.523840 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.524584 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.524648 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:05.524706 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:05.525073 (+ 367us) server_negotiation.cc:299] Negotiation successful
0925 13:08:05.525117 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":78,"threads_started":1}
I20260925 13:08:05.525236 11197 tablet_service.cc:1467] Processing CreateTablet for tablet 3d364670f38d4e4b948c7e44d1a0843b (DEFAULT_TABLE table=test-table [id=f0271c385774405f9d8460a82e26d17e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:08:05.525501 11197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d364670f38d4e4b948c7e44d1a0843b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:08:05.527386 11331 tablet_service.cc:1467] Processing CreateTablet for tablet 3d364670f38d4e4b948c7e44d1a0843b (DEFAULT_TABLE table=test-table [id=f0271c385774405f9d8460a82e26d17e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:08:05.527707 11331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d364670f38d4e4b948c7e44d1a0843b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:08:05.527997 11545 tablet_bootstrap.cc:492] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede: Bootstrap starting.
I20260925 13:08:05.528184 11546 tablet_bootstrap.cc:492] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18: Bootstrap starting.
I20260925 13:08:05.528584 11545 tablet_bootstrap.cc:654] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede: Neither blocks nor log segments found. Creating new log.
I20260925 13:08:05.528771 11546 tablet_bootstrap.cc:654] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18: Neither blocks nor log segments found. Creating new log.
I20260925 13:08:05.528990 11545 log.cc:824] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede: Log is configured to *not* fsync() on all Append() calls
I20260925 13:08:05.529331 11546 log.cc:824] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18: Log is configured to *not* fsync() on all Append() calls
I20260925 13:08:05.529665 11545 tablet_bootstrap.cc:492] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede: No bootstrap required, opened a new log
I20260925 13:08:05.529726 11545 ts_tablet_manager.cc:1397] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:08:05.530045 11546 tablet_bootstrap.cc:492] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18: No bootstrap required, opened a new log
I20260925 13:08:05.530099 11546 ts_tablet_manager.cc:1397] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:08:05.531005 11549 tablet_bootstrap.cc:492] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e: Bootstrap starting.
I20260925 13:08:05.531618 11549 tablet_bootstrap.cc:654] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e: Neither blocks nor log segments found. Creating new log.
I20260925 13:08:05.531478 11545 raft_consensus.cc:348] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb1d2b3a4f554104a4b5869b50f13ede" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43429 } } peers { permanent_uuid: "79d3562a70b04881835ece829a4fad5e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42055 } } peers { permanent_uuid: "aa78d57eef9d4be5aff427dc29d9ea18" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43123 } }
I20260925 13:08:05.531664 11545 raft_consensus.cc:374] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:08:05.531702 11545 raft_consensus.cc:729] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb1d2b3a4f554104a4b5869b50f13ede, State: Initialized, Role: FOLLOWER
I20260925 13:08:05.531890 11545 consensus_queue.cc:260] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb1d2b3a4f554104a4b5869b50f13ede" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43429 } } peers { permanent_uuid: "79d3562a70b04881835ece829a4fad5e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42055 } } peers { permanent_uuid: "aa78d57eef9d4be5aff427dc29d9ea18" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43123 } }
I20260925 13:08:05.531865 11546 raft_consensus.cc:348] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb1d2b3a4f554104a4b5869b50f13ede" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43429 } } peers { permanent_uuid: "79d3562a70b04881835ece829a4fad5e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42055 } } peers { permanent_uuid: "aa78d57eef9d4be5aff427dc29d9ea18" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43123 } }
I20260925 13:08:05.532029 11546 raft_consensus.cc:374] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:08:05.532087 11546 raft_consensus.cc:729] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa78d57eef9d4be5aff427dc29d9ea18, State: Initialized, Role: FOLLOWER
I20260925 13:08:05.532186 11545 ts_tablet_manager.cc:1428] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 13:08:05.532186 11549 log.cc:824] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e: Log is configured to *not* fsync() on all Append() calls
I20260925 13:08:05.532204 11546 consensus_queue.cc:260] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb1d2b3a4f554104a4b5869b50f13ede" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43429 } } peers { permanent_uuid: "79d3562a70b04881835ece829a4fad5e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42055 } } peers { permanent_uuid: "aa78d57eef9d4be5aff427dc29d9ea18" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43123 } }
I20260925 13:08:05.532328 11531 heartbeater.cc:503] Master 127.0.206.126:39503 was elected leader, sending a full tablet report...
I20260925 13:08:05.532495 11263 heartbeater.cc:503] Master 127.0.206.126:39503 was elected leader, sending a full tablet report...
I20260925 13:08:05.532483 11546 ts_tablet_manager.cc:1428] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 13:08:05.532907 11549 tablet_bootstrap.cc:492] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e: No bootstrap required, opened a new log
I20260925 13:08:05.532970 11549 ts_tablet_manager.cc:1397] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:08:05.534461 11549 raft_consensus.cc:348] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb1d2b3a4f554104a4b5869b50f13ede" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43429 } } peers { permanent_uuid: "79d3562a70b04881835ece829a4fad5e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42055 } } peers { permanent_uuid: "aa78d57eef9d4be5aff427dc29d9ea18" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43123 } }
I20260925 13:08:05.534586 11549 raft_consensus.cc:374] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:08:05.534607 11549 raft_consensus.cc:729] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d3562a70b04881835ece829a4fad5e, State: Initialized, Role: FOLLOWER
I20260925 13:08:05.534765 11549 consensus_queue.cc:260] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb1d2b3a4f554104a4b5869b50f13ede" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43429 } } peers { permanent_uuid: "79d3562a70b04881835ece829a4fad5e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42055 } } peers { permanent_uuid: "aa78d57eef9d4be5aff427dc29d9ea18" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43123 } }
I20260925 13:08:05.535073 11549 ts_tablet_manager.cc:1428] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 13:08:05.535079 11397 heartbeater.cc:503] Master 127.0.206.126:39503 was elected leader, sending a full tablet report...
I20260925 13:08:05.549464 11551 raft_consensus.cc:482] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:08:05.549540 11551 raft_consensus.cc:504] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb1d2b3a4f554104a4b5869b50f13ede" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43429 } } peers { permanent_uuid: "79d3562a70b04881835ece829a4fad5e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42055 } } peers { permanent_uuid: "aa78d57eef9d4be5aff427dc29d9ea18" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43123 } }
I20260925 13:08:05.549922 11551 leader_election.cc:288] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb1d2b3a4f554104a4b5869b50f13ede (127.0.206.67:43429), 79d3562a70b04881835ece829a4fad5e (127.0.206.66:42055)
I20260925 13:08:05.553455 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.550135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:60487 (local address 127.0.206.66:42055)
0925 13:08:05.550209 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:05.550211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:05.550218 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:08:05.550310 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:05.550311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:05.550320 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:05.550397 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:08:05.550524 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.551230 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.552564 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.553119 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.553189 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:05.553196 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:05.553362 (+ 166us) server_negotiation.cc:299] Negotiation successful
0925 13:08:05.553394 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 13:08:05.553454 11541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.550062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:38077 (local address 127.0.206.67:43429)
0925 13:08:05.550100 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:05.550101 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:05.550109 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:08:05.550192 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:05.550194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:05.550204 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:05.550253 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:08:05.550408 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.551153 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.552564 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.553112 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.553188 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:05.553196 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:05.553352 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 13:08:05.553389 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 13:08:05.553494 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.550025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:43429 (local address 127.0.206.65:38077)
0925 13:08:05.550071 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:05.550082 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:05.550121 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:05.550258 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:05.550259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:05.550261 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:08:05.550323 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:05.550325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.551220 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.551220 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:05.552480 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:05.552482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.553197 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.553198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:05.553274 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:05.553276 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:08:05.553335 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 13:08:05.553399 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":89}
I20260925 13:08:05.553560 11554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.550038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:42055 (local address 127.0.206.65:60487)
0925 13:08:05.550176 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:05.550185 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:05.550229 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:05.550400 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:05.550401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:05.550403 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:08:05.550453 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:05.550454 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.551296 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.551297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:05.552481 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:05.552482 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.553197 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.553198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:05.553272 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:05.553277 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 13:08:05.553336 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 13:08:05.553399 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":6,"thread_start_us":78,"threads_started":1}
I20260925 13:08:05.553802 11485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d364670f38d4e4b948c7e44d1a0843b" candidate_uuid: "aa78d57eef9d4be5aff427dc29d9ea18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb1d2b3a4f554104a4b5869b50f13ede" is_pre_election: true
I20260925 13:08:05.553937 11485 raft_consensus.cc:2454] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa78d57eef9d4be5aff427dc29d9ea18 in term 0.
I20260925 13:08:05.553922 11351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d364670f38d4e4b948c7e44d1a0843b" candidate_uuid: "aa78d57eef9d4be5aff427dc29d9ea18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d3562a70b04881835ece829a4fad5e" is_pre_election: true
I20260925 13:08:05.554060 11351 raft_consensus.cc:2454] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa78d57eef9d4be5aff427dc29d9ea18 in term 0.
I20260925 13:08:05.554148 11153 leader_election.cc:302] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa78d57eef9d4be5aff427dc29d9ea18, bb1d2b3a4f554104a4b5869b50f13ede; no voters:
I20260925 13:08:05.554260 11551 raft_consensus.cc:2781] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:08:05.554316 11551 raft_consensus.cc:482] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:08:05.554359 11551 raft_consensus.cc:3037] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:08:05.555164 11551 raft_consensus.cc:504] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb1d2b3a4f554104a4b5869b50f13ede" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43429 } } peers { permanent_uuid: "79d3562a70b04881835ece829a4fad5e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42055 } } peers { permanent_uuid: "aa78d57eef9d4be5aff427dc29d9ea18" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43123 } }
I20260925 13:08:05.555289 11551 leader_election.cc:288] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [CANDIDATE]: Term 1 election: Requested vote from peers bb1d2b3a4f554104a4b5869b50f13ede (127.0.206.67:43429), 79d3562a70b04881835ece829a4fad5e (127.0.206.66:42055)
I20260925 13:08:05.555414 11485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d364670f38d4e4b948c7e44d1a0843b" candidate_uuid: "aa78d57eef9d4be5aff427dc29d9ea18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb1d2b3a4f554104a4b5869b50f13ede"
I20260925 13:08:05.555485 11485 raft_consensus.cc:3037] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:08:05.555478 11351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d364670f38d4e4b948c7e44d1a0843b" candidate_uuid: "aa78d57eef9d4be5aff427dc29d9ea18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d3562a70b04881835ece829a4fad5e"
I20260925 13:08:05.555527 11351 raft_consensus.cc:3037] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:08:05.556062 11485 raft_consensus.cc:2454] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa78d57eef9d4be5aff427dc29d9ea18 in term 1.
I20260925 13:08:05.556069 11351 raft_consensus.cc:2454] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa78d57eef9d4be5aff427dc29d9ea18 in term 1.
I20260925 13:08:05.556218 11153 leader_election.cc:302] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa78d57eef9d4be5aff427dc29d9ea18, bb1d2b3a4f554104a4b5869b50f13ede; no voters:
I20260925 13:08:05.556311 11551 raft_consensus.cc:2781] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:08:05.556433 11551 raft_consensus.cc:686] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [term 1 LEADER]: Becoming Leader. State: Replica: aa78d57eef9d4be5aff427dc29d9ea18, State: Running, Role: LEADER
I20260925 13:08:05.556535 11551 consensus_queue.cc:237] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb1d2b3a4f554104a4b5869b50f13ede" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43429 } } peers { permanent_uuid: "79d3562a70b04881835ece829a4fad5e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42055 } } peers { permanent_uuid: "aa78d57eef9d4be5aff427dc29d9ea18" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43123 } }
I20260925 13:08:05.557286 11071 catalog_manager.cc:5495] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa78d57eef9d4be5aff427dc29d9ea18 (127.0.206.65). New cstate: current_term: 1 leader_uuid: "aa78d57eef9d4be5aff427dc29d9ea18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb1d2b3a4f554104a4b5869b50f13ede" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79d3562a70b04881835ece829a4fad5e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa78d57eef9d4be5aff427dc29d9ea18" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43123 } health_report { overall_health: HEALTHY } } }
I20260925 13:08:05.568620 11544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.566373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45426 (local address 127.0.206.65:43123)
0925 13:08:05.566427 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:05.566429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:05.566438 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:08:05.566495 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:05.566495 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:05.566508 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:05.566565 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:08:05.566690 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.567243 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.567985 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:05.568160 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:05.568211 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:05.568366 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:08:05.568449 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:08:05.568506 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:05.568543 (+ 37us) server_negotiation.cc:299] Negotiation successful
0925 13:08:05.568558 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 13:08:05.570057 11351 raft_consensus.cc:1261] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e [term 1 FOLLOWER]: Refusing update from remote peer aa78d57eef9d4be5aff427dc29d9ea18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:08:05.570076 11485 raft_consensus.cc:1261] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede [term 1 FOLLOWER]: Refusing update from remote peer aa78d57eef9d4be5aff427dc29d9ea18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:08:05.570369 11551 consensus_queue.cc:1035] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d3562a70b04881835ece829a4fad5e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:08:05.570524 11555 consensus_queue.cc:1035] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb1d2b3a4f554104a4b5869b50f13ede" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:08:05.572728 11558 mvcc.cc:205] Tried to move back new op lower bound from 7333239544092897280 to 7333239544040226816. Current Snapshot: MvccSnapshot[applied={T|T < 7333239544092897280}]
I20260925 13:08:05.573050 11559 mvcc.cc:205] Tried to move back new op lower bound from 7333239544092897280 to 7333239544040226816. Current Snapshot: MvccSnapshot[applied={T|T < 7333239544092897280}]
I20260925 13:08:05.573721 11560 mvcc.cc:205] Tried to move back new op lower bound from 7333239544092897280 to 7333239544040226816. Current Snapshot: MvccSnapshot[applied={T|T < 7333239544092897280}]
I20260925 13:08:05.574922 11071 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:08:05.574992 11071 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:08:05.575682 11071 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f090d52c84a4b51bde0cde0834da743: Sending DeleteTablet for 3 replicas of tablet 3d364670f38d4e4b948c7e44d1a0843b
I20260925 13:08:05.575992 11331 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d364670f38d4e4b948c7e44d1a0843b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:08:05 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:57978
I20260925 13:08:05.576001 11197 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d364670f38d4e4b948c7e44d1a0843b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:08:05 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:45412
I20260925 13:08:05.576262 11569 tablet_replica.cc:330] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18: stopping tablet replica
I20260925 13:08:05.576272 11465 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d364670f38d4e4b948c7e44d1a0843b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:08:05 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:41016
I20260925 13:08:05.576396 11569 raft_consensus.cc:2229] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:08:05.576550 11569 raft_consensus.cc:2258] T 3d364670f38d4e4b948c7e44d1a0843b P aa78d57eef9d4be5aff427dc29d9ea18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:08:05.576588 11570 tablet_replica.cc:330] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede: stopping tablet replica
I20260925 13:08:05.576687 11570 raft_consensus.cc:2229] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:08:05.576761 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 11134
I20260925 13:08:05.578223 11570 raft_consensus.cc:2258] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:08:05.578456 11568 tablet_replica.cc:330] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e: stopping tablet replica
I20260925 13:08:05.578619 11568 raft_consensus.cc:2229] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:08:05.578677 11570 ts_tablet_manager.cc:1905] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:08:05.578774 11568 raft_consensus.cc:2258] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:08:05.579164 11568 ts_tablet_manager.cc:1905] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:08:05.580320 11570 ts_tablet_manager.cc:1918] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:08:05.580389 11570 log.cc:1196] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/3d364670f38d4e4b948c7e44d1a0843b
I20260925 13:08:05.580658 11570 ts_tablet_manager.cc:1939] T 3d364670f38d4e4b948c7e44d1a0843b P bb1d2b3a4f554104a4b5869b50f13ede: Deleting consensus metadata
I20260925 13:08:05.580808 11568 ts_tablet_manager.cc:1918] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:08:05.580873 11568 log.cc:1196] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/3d364670f38d4e4b948c7e44d1a0843b
I20260925 13:08:05.581043 11059 catalog_manager.cc:4841] TS bb1d2b3a4f554104a4b5869b50f13ede (127.0.206.67:43429): tablet 3d364670f38d4e4b948c7e44d1a0843b (table test-table [id=f0271c385774405f9d8460a82e26d17e]) successfully deleted
I20260925 13:08:05.581194 11568 ts_tablet_manager.cc:1939] T 3d364670f38d4e4b948c7e44d1a0843b P 79d3562a70b04881835ece829a4fad5e: Deleting consensus metadata
I20260925 13:08:05.581570 11058 catalog_manager.cc:4841] TS 79d3562a70b04881835ece829a4fad5e (127.0.206.66:42055): tablet 3d364670f38d4e4b948c7e44d1a0843b (table test-table [id=f0271c385774405f9d8460a82e26d17e]) successfully deleted
W20260925 13:08:05.582098 11058 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:43123 (error 108)
I20260925 13:08:05.582341 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 11268
I20260925 13:08:05.582916 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:05.582444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:43123 (local address 127.0.0.1:45430)
0925 13:08:05.582836 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:05.582880 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:43123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368}
W20260925 13:08:05.583778 11058 catalog_manager.cc:4584] TS aa78d57eef9d4be5aff427dc29d9ea18 (127.0.206.65:43123): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3d364670f38d4e4b948c7e44d1a0843b: Network error: Client connection negotiation failed: client connection to 127.0.206.65:43123: connect: Connection refused (error 111)
I20260925 13:08:05.586786 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 11402
I20260925 13:08:05.591979 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 11040
2026-09-25T13:08:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1867 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:08:05 dist-test-slave-chkf krb5kdc[11574](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:08:05 dist-test-slave-chkf krb5kdc[11574](info): set up 2 sockets
Sep 25 13:08:05 dist-test-slave-chkf krb5kdc[11574](info): commencing operation
krb5kdc: starting...
W20260925 13:08:06.611642 825 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:08:06 dist-test-slave-chkf krb5kdc[11574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:08:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:08:06Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.206.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.206.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/rangeradmin_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/rangeradmin_127.0.206.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.206.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.206.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/rangerlookup_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/rangerlookup_127.0.206.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.206.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.206.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/HTTP_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/HTTP_127.0.206.84@KRBTEST.COM.keytab.
I20260925 13:08:06.763787 825 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskG_Ts_H/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/postgres -l logfile start
2026-09-25 13:08:09.227 UTC [11604] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 13:08:09.227 UTC [11604] LOG: listening on IPv4 address "127.0.206.84", port 35981
2026-09-25 13:08:09.230 UTC [11604] LOG: listening on Unix socket "/tmp/.s.PGSQL.35981"
2026-09-25 13:08:09.262 UTC [11607] LOG: database system was shut down at 2026-09-25 13:08:07 UTC
2026-09-25 13:08:09.265 UTC [11604] LOG: database system is ready to accept connections
I20260925 13:08:10.216699 825 mini_postgres.cc:96] Postgres bound to 35981
2026-09-25 13:08:10.222 UTC [11615] FATAL: database "slave" does not exist
127.0.206.84:35981 - accepting connections
I20260925 13:08:10.223333 825 mini_ranger.cc:161] Starting Ranger...
I20260925 13:08:10.237357 825 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 13:08:10.709772 825 mini_ranger.cc:87] Created ranger Postgres database
I20260925 13:08:10.709858 825 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ranger-admin
2026-09-25 13:08:10,980 [I] DB FLAVOR :POSTGRES
2026-09-25 13:08:10,981 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 13:08:10,981 [I] Checking connection..
2026-09-25 13:08:10,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 13:08:11,203 [I] Checking connection passed.
2026-09-25 13:08:11,204 [I] --------- Verifying version history table ---------
2026-09-25 13:08:11,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 13:08:11,427 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 13:08:11,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 13:08:11,671 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 13:08:11,671 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 13:08:11,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 13:08:11.921 UTC [11679] WARNING: there is no transaction in progress
2026-09-25 13:08:11,927 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 13:08:11,927 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 13:08:12,180 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 13:08:12,180 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 13:08:12,180 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 13:08:12,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 13:08:12,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-chkf.c.gcp-upstream.internal','N') ;"
2026-09-25 13:08:12,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 13:08:13,106 [I] Table x_portal_user does not exist in database ranger
2026-09-25 13:08:13,107 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 13:08:13,343 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 13:08:13,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 13:08:13,579 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 13:08:13,814 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 13:08:13,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 13:08:14.734 UTC [11787] WARNING: there is no transaction in progress
2026-09-25 13:08:14.749 UTC [11787] WARNING: there is no transaction in progress
2026-09-25 13:08:14.764 UTC [11787] WARNING: there is no transaction in progress
2026-09-25 13:08:14.778 UTC [11787] WARNING: there is no transaction in progress
2026-09-25 13:08:14.793 UTC [11787] WARNING: there is no transaction in progress
2026-09-25 13:08:14.809 UTC [11787] WARNING: there is no transaction in progress
2026-09-25 13:08:14.862 UTC [11787] WARNING: there is no transaction in progress
2026-09-25 13:08:14.869 UTC [11787] WARNING: there is no transaction in progress
2026-09-25 13:08:14.883 UTC [11787] WARNING: there is no transaction in progress
2026-09-25 13:08:14.893 UTC [11787] WARNING: there is no transaction in progress
2026-09-25 13:08:14.902 UTC [11787] WARNING: there is no transaction in progress
2026-09-25 13:08:14.915 UTC [11787] WARNING: there is no transaction in progress
2026-09-25 13:08:14.925 UTC [11787] WARNING: there is no transaction in progress
2026-09-25 13:08:14.931 UTC [11787] WARNING: there is no transaction in progress
2026-09-25 13:08:14.938 UTC [11787] WARNING: there is no transaction in progress
2026-09-25 13:08:14.945 UTC [11787] WARNING: there is no transaction in progress
2026-09-25 13:08:15,573 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 13:08:15,573 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 13:08:15,820 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 13:08:15,821 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 13:08:16,074 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 13:08:16,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 13:08:16,324 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 13:08:16,325 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 13:08:16,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 13:08:16,787 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-chkf.c.gcp-upstream.internal';"
2026-09-25 13:08:16,998 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 13:08:16,999 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 13:08:17,249 [I] DB_PATCHES have already been applied
I20260925 13:08:17.254494 825 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ranger-admin:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskG_Ts_H/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 13:08:17.254549 825 mini_ranger.cc:193] Using host: 127.0.206.84
I20260925 13:08:17.257525 825 mini_ranger.cc:227] Ranger admin URL: http://127.0.206.84:40901
Sep 25, 2026 1:08:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 1:08:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 1:08:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 1:08:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 1:08:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 1:08:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 1:08:18 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 13:08:18.369190 825 mini_ranger.cc:160] Time spent starting Ranger: real 8.146s user 0.000s sys 0.009s
Sep 25, 2026 1:08:36 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 1:08:36 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 25, 2026 1:08:36 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 1:08:36 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 1:08:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 1:08:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 1:08:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 1:08:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 1:08:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 1:08:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 1:08:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 1:08:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 1:08:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 1:08:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 1:08:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 1:08:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 1:08:36 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 1:08:37 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 13:08:38.193001 825 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.206.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.206.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/rangerkms_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/rangerkms_127.0.206.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.206.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.206.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.206.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/HTTP_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.206.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/HTTP_127.0.206.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 13:08:38 dist-test-slave-chkf krb5kdc[11574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341718, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 13:08:38.282871 825 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 13:08:38.292155 825 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 13:08:39.180217 825 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 13:08:39.180301 825 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ranger-kms
I20260925 13:08:39.180325 825 mini_ranger_kms.cc:227] Using postgres at 127.0.206.84:35981
2026-09-25 13:08:39,368 [I] DB FLAVOR :POSTGRES
2026-09-25 13:08:39,368 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 13:08:39,369 [I] Checking connection
2026-09-25 13:08:39,369 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 13:08:39,619 [I] connection success
2026-09-25 13:08:39,620 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 13:08:39,620 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 13:08:39,620 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 13:08:39,847 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 13:08:39,848 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 13:08:39,848 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 13:08:39,848 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:35981/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 13:08:40,102 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 13:08:40.558534 825 mini_ranger_kms.cc:310] Created kms service
I20260925 13:08:40.764251 825 mini_ranger_kms.cc:326] Created kudu user
I20260925 13:08:40.804847 825 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 13:08:40 dist-test-slave-chkf krb5kdc[11574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341718, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260925 13:08:41.181239 825 mini_ranger_kms.cc:384] Added ranger policy
I20260925 13:08:41.181352 825 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ranger-kms:/tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 13:08:41.181380 825 mini_ranger_kms.cc:240] Using host: 127.0.206.84
I20260925 13:08:41.184583 825 mini_ranger_kms.cc:276] Ranger KMS PID: 11967
I20260925 13:08:41.184695 825 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.206.84:47081
Sep 25, 2026 1:08:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 1:08:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 1:08:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 1:08:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 1:08:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 1:08:42 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260925 13:08:43.369514 825 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.087s user 0.001s sys 0.169s
I20260925 13:08:43.369622 825 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 13:08:43.369709 825 mini_ranger_kms.cc:401] 127.0.206.84:47081/kms/v1/keys
Sep 25 13:08:43 dist-test-slave-chkf krb5kdc[11574](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790341723, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.206.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 13:08:43 dist-test-slave-chkf krb5kdc[11574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341723, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.206.84@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
Sep 25 13:08:44 dist-test-slave-chkf krb5kdc[11574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341723, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.206.84@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
Sep 25 13:08:46 dist-test-slave-chkf krb5kdc[11574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:08:46.325407 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43949
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:43949
--ranger_config_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:47081/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:08:46.409449 12018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:08:46.409642 12018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:08:46.409672 12018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:08:46.410871 12018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:08:46.410910 12018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:08:46.410924 12018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:08:46.410938 12018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:08:46.410986 12018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:08:46.412447 12018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.206.84:47081/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:43949
--ranger_config_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:08:46.412686 12018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:08:46.412923 12018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:08:46.417657 12024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:46.417783 12026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:46.417667 12023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:46.418409 12018 server_base.cc:1034] running on GCE node
I20260925 13:08:46.418680 12018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:08:46.419047 12018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:08:46.420214 12018 hybrid_clock.cc:648] HybridClock initialized: now 1790341726420188 us; error 40 us; skew 500 ppm
Sep 25 13:08:46 dist-test-slave-chkf krb5kdc[11574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341726, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:08:46.426777 12018 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:08:46.427266 12018 webserver.cc:460] Webserver started at http://127.0.206.126:40221/ using document root <none> and password file <none>
I20260925 13:08:46.427505 12018 fs_manager.cc:359] Metadata directory not provided
I20260925 13:08:46.427565 12018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:08:46.427726 12018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 13:08:46 dist-test-slave-chkf krb5kdc[11574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341726, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260925 13:08:46.479250 12018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "b418cdd7906d49d8bde8b5b1a68cbed6"
format_stamp: "Formatted at 2026-09-25 13:08:46 on dist-test-slave-chkf"
server_key: "02707db6ff5a57f6705ec06061df7cef"
server_key_iv: "ff45e87d8476a6da10ebfef34d133c86"
server_key_version: "kuduclusterkey@0"
I20260925 13:08:46.480115 12018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "b418cdd7906d49d8bde8b5b1a68cbed6"
format_stamp: "Formatted at 2026-09-25 13:08:46 on dist-test-slave-chkf"
server_key: "02707db6ff5a57f6705ec06061df7cef"
server_key_iv: "ff45e87d8476a6da10ebfef34d133c86"
server_key_version: "kuduclusterkey@0"
I20260925 13:08:46.481654 12018 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:08:46.496366 12033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:46.496615 12018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:08:46.496721 12018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "b418cdd7906d49d8bde8b5b1a68cbed6"
format_stamp: "Formatted at 2026-09-25 13:08:46 on dist-test-slave-chkf"
server_key: "02707db6ff5a57f6705ec06061df7cef"
server_key_iv: "ff45e87d8476a6da10ebfef34d133c86"
server_key_version: "kuduclusterkey@0"
I20260925 13:08:46.496819 12018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:08:46.521133 12018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:08:46.523531 12018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:08:46.523711 12018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:08:46.529204 12018 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:43949
I20260925 13:08:46.529379 12085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:43949 every 8 connection(s)
I20260925 13:08:46.529659 12018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:08:46.530380 12086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:08:46.532282 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 12018
I20260925 13:08:46.532385 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:08:46.532585 825 external_mini_cluster.cc:1428] Setting key 285a579cd5707ddc5a74ea4a4bf556c5
I20260925 13:08:46.533779 12086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6: Bootstrap starting.
I20260925 13:08:46.534525 12086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6: Neither blocks nor log segments found. Creating new log.
I20260925 13:08:46.534891 12086 log.cc:824] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6: Log is configured to *not* fsync() on all Append() calls
I20260925 13:08:46.535686 12086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6: No bootstrap required, opened a new log
I20260925 13:08:46.537882 12086 raft_consensus.cc:348] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b418cdd7906d49d8bde8b5b1a68cbed6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43949 } }
I20260925 13:08:46.538048 12086 raft_consensus.cc:374] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:08:46.538085 12086 raft_consensus.cc:729] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b418cdd7906d49d8bde8b5b1a68cbed6, State: Initialized, Role: FOLLOWER
I20260925 13:08:46.538273 12086 consensus_queue.cc:260] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b418cdd7906d49d8bde8b5b1a68cbed6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43949 } }
I20260925 13:08:46.538368 12086 raft_consensus.cc:388] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:08:46.538403 12086 raft_consensus.cc:482] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:08:46.538482 12086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:08:46.539163 12086 raft_consensus.cc:504] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b418cdd7906d49d8bde8b5b1a68cbed6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43949 } }
I20260925 13:08:46.539286 12086 leader_election.cc:302] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b418cdd7906d49d8bde8b5b1a68cbed6; no voters:
I20260925 13:08:46.539502 12086 leader_election.cc:288] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:08:46.539554 12091 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:08:46.539855 12086 sys_catalog.cc:564] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6 [sys.catalog]: configured and running, proceeding with master startup.
Sep 25 13:08:46 dist-test-slave-chkf krb5kdc[11574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:08:46.541811 12091 raft_consensus.cc:686] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6 [term 1 LEADER]: Becoming Leader. State: Replica: b418cdd7906d49d8bde8b5b1a68cbed6, State: Running, Role: LEADER
I20260925 13:08:46.541949 12091 consensus_queue.cc:237] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b418cdd7906d49d8bde8b5b1a68cbed6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43949 } }
I20260925 13:08:46.542498 12086 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 13:08:46.543184 12103 sys_catalog.cc:455] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b418cdd7906d49d8bde8b5b1a68cbed6. Latest consensus state: current_term: 1 leader_uuid: "b418cdd7906d49d8bde8b5b1a68cbed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b418cdd7906d49d8bde8b5b1a68cbed6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43949 } } }
I20260925 13:08:46.543282 12103 sys_catalog.cc:458] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6 [sys.catalog]: This master's current role is: LEADER
I20260925 13:08:46.543476 12092 sys_catalog.cc:455] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b418cdd7906d49d8bde8b5b1a68cbed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b418cdd7906d49d8bde8b5b1a68cbed6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43949 } } }
I20260925 13:08:46.543490 12108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:08:46.543573 12092 sys_catalog.cc:458] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6 [sys.catalog]: This master's current role is: LEADER
I20260925 13:08:46.547600 12108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:08:46.547605 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:46.533144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49652 (local address 127.0.206.126:43949)
0925 13:08:46.533327 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:46.533337 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:46.533372 (+ 35us) server_negotiation.cc:407] Connection header received
0925 13:08:46.533902 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:46.533912 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:46.534004 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:46.534161 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:46.534926 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:46.536244 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:46.537304 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:46.537999 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:46.538112 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:46.541235 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:08:46.541239 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:08:46.541240 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:08:46.541252 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:08:46.543280 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:46.543729 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:46.543732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:46.543732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:46.543837 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:46.544903 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:46.544935 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:46.544935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:46.546601 (+ 1666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:08:46.546859 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:46.546946 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 13:08:46.547237 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":70,"threads_started":1}
I20260925 13:08:46.551186 12108 catalog_manager.cc:1347] Generated new cluster ID: a4629aac5d6b4ae4ad88e9db7bc95f74
I20260925 13:08:46.551244 12108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:08:46.574415 12108 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:08:46.575361 12108 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:08:46.578537 12108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6: Generated new TSK 0
I20260925 13:08:46.578804 12108 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260925 13:08:47.321831 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:47081/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43949
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:08:47.440367 12132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:08:47.441298 12132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:08:47.441461 12132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:08:47.443902 12132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:08:47.444006 12132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:08:47.444092 12132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:08:47.447127 12132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.206.84:47081/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:08:47.447403 12132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:08:47.447631 12132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:08:47.453064 12137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:47.453073 12138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:47.453349 12140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:47.453636 12132 server_base.cc:1034] running on GCE node
I20260925 13:08:47.453891 12132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:08:47.454137 12132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:08:47.455330 12132 hybrid_clock.cc:648] HybridClock initialized: now 1790341727455316 us; error 26 us; skew 500 ppm
Sep 25 13:08:47 dist-test-slave-chkf krb5kdc[11574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341727, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:08:47.457942 12132 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:08:47.458425 12132 webserver.cc:460] Webserver started at http://127.0.206.65:45765/ using document root <none> and password file <none>
I20260925 13:08:47.458681 12132 fs_manager.cc:359] Metadata directory not provided
I20260925 13:08:47.458796 12132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:08:47.458961 12132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 13:08:47 dist-test-slave-chkf krb5kdc[11574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341727, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260925 13:08:47.473182 12132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "1ec51e6cf69947479a79e2c91c12c995"
format_stamp: "Formatted at 2026-09-25 13:08:47 on dist-test-slave-chkf"
server_key: "20cd488aa0936d9b95dba93e90d5aab9"
server_key_iv: "3619f91010a83f944edbb77225d238b2"
server_key_version: "kuduclusterkey@0"
I20260925 13:08:47.473596 12132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "1ec51e6cf69947479a79e2c91c12c995"
format_stamp: "Formatted at 2026-09-25 13:08:47 on dist-test-slave-chkf"
server_key: "20cd488aa0936d9b95dba93e90d5aab9"
server_key_iv: "3619f91010a83f944edbb77225d238b2"
server_key_version: "kuduclusterkey@0"
I20260925 13:08:47.475716 12132 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260925 13:08:47.487015 12147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:47.487228 12132 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:08:47.487340 12132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "1ec51e6cf69947479a79e2c91c12c995"
format_stamp: "Formatted at 2026-09-25 13:08:47 on dist-test-slave-chkf"
server_key: "20cd488aa0936d9b95dba93e90d5aab9"
server_key_iv: "3619f91010a83f944edbb77225d238b2"
server_key_version: "kuduclusterkey@0"
I20260925 13:08:47.487440 12132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:08:47.498322 12132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:08:47.500463 12132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:08:47.500643 12132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:08:47.500893 12132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:08:47.501255 12132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:08:47.501312 12132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:47.501349 12132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:08:47.501374 12132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:47.509596 12132 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:40301
I20260925 13:08:47.510008 12132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:08:47.510075 12260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:40301 every 8 connection(s)
I20260925 13:08:47.513935 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 12132
I20260925 13:08:47.514061 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:08:47.514264 825 external_mini_cluster.cc:1428] Setting key 0ae762a08ab947b1bff18314baff8093
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Sep 25 13:08:47 dist-test-slave-chkf krb5kdc[11574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341727, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
I20260925 13:08:47.534967 12264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:47.511276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:49279 (local address 127.0.206.126:43949)
0925 13:08:47.517606 (+ 6330us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:47.517609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:47.518628 (+ 1019us) server_negotiation.cc:407] Connection header received
0925 13:08:47.518658 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:47.518659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:47.518675 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:47.518766 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:47.519589 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:47.520463 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:47.522055 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:47.522399 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:47.522463 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:47.531433 (+ 8970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:08:47.531435 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:08:47.531435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:08:47.531444 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:08:47.533178 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:47.533757 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:47.533758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:47.533759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:47.533821 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:47.534117 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:47.534117 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:47.534118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:47.534263 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:08:47.534330 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:47.534773 (+ 443us) server_negotiation.cc:299] Negotiation successful
0925 13:08:47.534862 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6294,"thread_start_us":94,"threads_started":1}
I20260925 13:08:47.535679 12263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:47.511272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43949 (local address 127.0.206.65:49279)
0925 13:08:47.517675 (+ 6403us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:47.517723 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:47.518482 (+ 759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:47.518914 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:47.518920 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:47.519192 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:47.519480 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:47.519487 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:47.520578 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:47.520579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:47.521955 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:47.521959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:47.522538 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:47.522539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:47.522690 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:47.523388 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:08:47.523399 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:47.531303 (+ 7904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:08:47.533291 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:08:47.533301 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:08:47.533303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:08:47.533659 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:08:47.533897 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:08:47.533899 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:08:47.533899 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:08:47.534038 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:08:47.534359 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:08:47.534366 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:08:47.534534 (+ 168us) client_negotiation.cc:769] Sending connection context
0925 13:08:47.534659 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 13:08:47.535440 (+ 781us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6359,"thread_start_us":88,"threads_started":1}
I20260925 13:08:47.536351 12261 heartbeater.cc:347] Connected to a master server at 127.0.206.126:43949
I20260925 13:08:47.536440 12261 heartbeater.cc:464] Registering TS with master...
I20260925 13:08:47.536608 12261 heartbeater.cc:511] Master 127.0.206.126:43949 requested a full tablet report, sending...
I20260925 13:08:47.537092 12050 ts_manager.cc:194] Registered new tserver with Master: 1ec51e6cf69947479a79e2c91c12c995 (127.0.206.65:40301)
I20260925 13:08:47.537896 12050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:49279
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260925 13:08:47.553428 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:47081/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43949
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:08:47.684984 12269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:08:47.685289 12269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:08:47.685326 12269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:08:47.687269 12269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:08:47.687335 12269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:08:47.687376 12269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:08:47.689790 12269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.206.84:47081/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:08:47.690055 12269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:08:47.690390 12269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:08:47.696863 12277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:47.696870 12276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:47.697244 12279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:47.697387 12269 server_base.cc:1034] running on GCE node
I20260925 13:08:47.697678 12269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:08:47.697937 12269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:08:47.699100 12269 hybrid_clock.cc:648] HybridClock initialized: now 1790341727699073 us; error 41 us; skew 500 ppm
Sep 25 13:08:47 dist-test-slave-chkf krb5kdc[11574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341727, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:08:47.702500 12269 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:08:47.703079 12269 webserver.cc:460] Webserver started at http://127.0.206.66:35399/ using document root <none> and password file <none>
I20260925 13:08:47.703365 12269 fs_manager.cc:359] Metadata directory not provided
I20260925 13:08:47.703439 12269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:08:47.703583 12269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 13:08:47 dist-test-slave-chkf krb5kdc[11574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341727, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260925 13:08:47.717864 12269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "f0fe2efdf1ac42adba83ecdce3fe2b18"
format_stamp: "Formatted at 2026-09-25 13:08:47 on dist-test-slave-chkf"
server_key: "ec56c001d70416bae10f66f1504573aa"
server_key_iv: "afee109ae2aa1f89062234b0c45fc17b"
server_key_version: "kuduclusterkey@0"
I20260925 13:08:47.718526 12269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "f0fe2efdf1ac42adba83ecdce3fe2b18"
format_stamp: "Formatted at 2026-09-25 13:08:47 on dist-test-slave-chkf"
server_key: "ec56c001d70416bae10f66f1504573aa"
server_key_iv: "afee109ae2aa1f89062234b0c45fc17b"
server_key_version: "kuduclusterkey@0"
I20260925 13:08:47.722018 12269 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.001s
Sep 25 13:08:47 dist-test-slave-chkf krb5kdc[11574](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790341727, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:08:47.762856 12286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:47.763087 12269 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.001s sys 0.000s
I20260925 13:08:47.763212 12269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "f0fe2efdf1ac42adba83ecdce3fe2b18"
format_stamp: "Formatted at 2026-09-25 13:08:47 on dist-test-slave-chkf"
server_key: "ec56c001d70416bae10f66f1504573aa"
server_key_iv: "afee109ae2aa1f89062234b0c45fc17b"
server_key_version: "kuduclusterkey@0"
I20260925 13:08:47.763342 12269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:08:47.791004 12269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:08:47.793581 12269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:08:47.793762 12269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:08:47.794026 12269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:08:47.794406 12269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:08:47.794468 12269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:47.794507 12269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:08:47.794531 12269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:47.802345 12269 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:46411
I20260925 13:08:47.807348 12399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:46411 every 8 connection(s)
I20260925 13:08:47.807602 12269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
Sep 25 13:08:47 dist-test-slave-chkf krb5kdc[11574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341727, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:08:47.815601 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 12269
I20260925 13:08:47.815725 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:08:47.815920 825 external_mini_cluster.cc:1428] Setting key c67cea2bfd2e3c90cb254cdb7a6f5980
I20260925 13:08:47.820533 12264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:47.808666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:56587 (local address 127.0.206.126:43949)
0925 13:08:47.808711 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:47.808714 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:47.808980 (+ 266us) server_negotiation.cc:407] Connection header received
0925 13:08:47.809661 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:47.809663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:47.809679 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:47.809711 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:47.810532 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:47.811297 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:47.812556 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:47.812770 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:47.812860 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:47.814979 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:08:47.814981 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:08:47.814981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:08:47.814989 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:08:47.818944 (+ 3955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:47.819450 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:47.819451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:47.819451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:47.819512 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:47.819786 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:47.819787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:47.819787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:47.819919 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:08:47.819961 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:47.820345 (+ 384us) server_negotiation.cc:299] Negotiation successful
0925 13:08:47.820435 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 13:08:47.821273 12403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:47.808650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43949 (local address 127.0.206.66:56587)
0925 13:08:47.808843 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:47.808893 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:47.809525 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:47.809901 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:47.809905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:47.810149 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:47.810417 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:47.810424 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:47.811413 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:47.811414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:47.812443 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:47.812445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:47.812868 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:47.812869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:47.812999 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:47.813547 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:08:47.813555 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:47.814887 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:08:47.819039 (+ 4152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:08:47.819047 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:08:47.819050 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:08:47.819359 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:08:47.819583 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:08:47.819584 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:08:47.819584 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:08:47.819712 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:08:47.820007 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:08:47.820013 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:08:47.820190 (+ 177us) client_negotiation.cc:769] Sending connection context
0925 13:08:47.820871 (+ 681us) client_negotiation.cc:240] Negotiation successful
0925 13:08:47.821032 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":107,"threads_started":1}
I20260925 13:08:47.821892 12401 heartbeater.cc:347] Connected to a master server at 127.0.206.126:43949
I20260925 13:08:47.821980 12401 heartbeater.cc:464] Registering TS with master...
I20260925 13:08:47.822163 12401 heartbeater.cc:511] Master 127.0.206.126:43949 requested a full tablet report, sending...
I20260925 13:08:47.822561 12050 ts_manager.cc:194] Registered new tserver with Master: f0fe2efdf1ac42adba83ecdce3fe2b18 (127.0.206.66:46411)
I20260925 13:08:47.823064 12050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:56587
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260925 13:08:47.853811 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:47081/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43949
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:08:48.018924 12406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:08:48.019304 12406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:08:48.019389 12406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:08:48.021265 12406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:08:48.021483 12406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:08:48.021571 12406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:08:48.023929 12406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.206.84:47081/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:08:48.024431 12406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:08:48.024741 12406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:08:48.031528 12412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:48.031549 12411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:08:48.031957 12414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:08:48.033041 12406 server_base.cc:1034] running on GCE node
I20260925 13:08:48.033304 12406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:08:48.033572 12406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:08:48.034780 12406 hybrid_clock.cc:648] HybridClock initialized: now 1790341728034765 us; error 28 us; skew 500 ppm
Sep 25 13:08:48 dist-test-slave-chkf krb5kdc[11574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341728, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:08:48.038120 12406 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:08:48.038646 12406 webserver.cc:460] Webserver started at http://127.0.206.67:45591/ using document root <none> and password file <none>
I20260925 13:08:48.038970 12406 fs_manager.cc:359] Metadata directory not provided
I20260925 13:08:48.039036 12406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:08:48.039160 12406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 13:08:48 dist-test-slave-chkf krb5kdc[11574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341728, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260925 13:08:48.053650 12406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "496f9417bbb14a6d851ccf5d67f88f08"
format_stamp: "Formatted at 2026-09-25 13:08:48 on dist-test-slave-chkf"
server_key: "4d87772a78f056b0822c963223bbe757"
server_key_iv: "96e0f96d0f35150c4b8216b5f888bf0f"
server_key_version: "kuduclusterkey@0"
I20260925 13:08:48.054106 12406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "496f9417bbb14a6d851ccf5d67f88f08"
format_stamp: "Formatted at 2026-09-25 13:08:48 on dist-test-slave-chkf"
server_key: "4d87772a78f056b0822c963223bbe757"
server_key_iv: "96e0f96d0f35150c4b8216b5f888bf0f"
server_key_version: "kuduclusterkey@0"
I20260925 13:08:48.056840 12406 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260925 13:08:48.070848 12421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:48.071018 12406 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260925 13:08:48.071106 12406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "496f9417bbb14a6d851ccf5d67f88f08"
format_stamp: "Formatted at 2026-09-25 13:08:48 on dist-test-slave-chkf"
server_key: "4d87772a78f056b0822c963223bbe757"
server_key_iv: "96e0f96d0f35150c4b8216b5f888bf0f"
server_key_version: "kuduclusterkey@0"
I20260925 13:08:48.071198 12406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:08:48.096900 12406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:08:48.099303 12406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:08:48.099434 12406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:08:48.099617 12406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:08:48.099952 12406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:08:48.099987 12406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:48.100015 12406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:08:48.100033 12406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:08:48.107502 12406 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:37965
I20260925 13:08:48.108021 12406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:08:48.108570 12534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:37965 every 8 connection(s)
I20260925 13:08:48.113365 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 12406
I20260925 13:08:48.113487 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:08:48.113689 825 external_mini_cluster.cc:1428] Setting key 67ad5d0052da7c9aa806bc180991cd7d
Sep 25 13:08:48 dist-test-slave-chkf krb5kdc[11574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341728, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:08:48.120796 12264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:48.109622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:40079 (local address 127.0.206.126:43949)
0925 13:08:48.109685 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:48.109687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:48.109888 (+ 201us) server_negotiation.cc:407] Connection header received
0925 13:08:48.110579 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:48.110581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:48.110593 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:48.110627 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:48.111589 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.112344 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.114285 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.114560 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.114603 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:48.117372 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:08:48.117373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:08:48.117374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:08:48.117381 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:08:48.119178 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:48.119698 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:48.119700 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:48.119700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:48.119758 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:48.120047 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:48.120048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:48.120049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:48.120183 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:08:48.120230 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:48.120602 (+ 372us) server_negotiation.cc:299] Negotiation successful
0925 13:08:48.120701 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 13:08:48.121529 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:48.109347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43949 (local address 127.0.206.67:40079)
0925 13:08:48.109755 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:48.109801 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:48.110445 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:48.110861 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:48.110866 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:48.111119 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:08:48.111473 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:48.111480 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.112458 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.112459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:48.114138 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:48.114140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.114820 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.114821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:48.114964 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:48.115587 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:08:48.115597 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:08:48.117259 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:08:48.119276 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:08:48.119283 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:08:48.119285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:08:48.119591 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:08:48.119841 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:08:48.119842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:08:48.119842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:08:48.119970 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:08:48.120282 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:08:48.120288 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:08:48.120446 (+ 158us) client_negotiation.cc:769] Sending connection context
0925 13:08:48.121137 (+ 691us) client_negotiation.cc:240] Negotiation successful
0925 13:08:48.121295 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":81,"threads_started":1}
I20260925 13:08:48.122175 12535 heartbeater.cc:347] Connected to a master server at 127.0.206.126:43949
I20260925 13:08:48.122264 12535 heartbeater.cc:464] Registering TS with master...
I20260925 13:08:48.122442 12535 heartbeater.cc:511] Master 127.0.206.126:43949 requested a full tablet report, sending...
I20260925 13:08:48.122896 12050 ts_manager.cc:194] Registered new tserver with Master: 496f9417bbb14a6d851ccf5d67f88f08 (127.0.206.67:37965)
I20260925 13:08:48.123370 12050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:40079
I20260925 13:08:48.125481 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:08:48.135201 12264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:48.127200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49654 (local address 127.0.206.126:43949)
0925 13:08:48.127237 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:48.127239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:48.127510 (+ 271us) server_negotiation.cc:407] Connection header received
0925 13:08:48.127605 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:48.127606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:48.127634 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:48.127672 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:08:48.128386 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.129130 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.130284 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.130552 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.130595 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:48.132166 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:08:48.132167 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:08:48.132167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:08:48.132174 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:08:48.133862 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:48.134248 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:48.134249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:48.134250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:48.134304 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:08:48.134507 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:08:48.134508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:08:48.134508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:08:48.134628 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:08:48.134759 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:48.135027 (+ 268us) server_negotiation.cc:299] Negotiation successful
0925 13:08:48.135102 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:08:48.137342 12050 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:08:48.139725 12050 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:08:48.149765 12552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:48.143298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:37965 (local address 127.0.0.1:42466)
0925 13:08:48.144364 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:48.144403 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:48.144468 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:48.145000 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:48.145003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:48.145005 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:08:48.145066 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:48.145068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.146792 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.146793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:48.148119 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:48.148122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.149456 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.149457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:48.149536 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:48.149548 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 13:08:48.149616 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 13:08:48.149678 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1034,"thread_start_us":64,"threads_started":1}
I20260925 13:08:48.150346 12551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:48.143720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42466 (local address 127.0.206.67:37965)
0925 13:08:48.144740 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:48.144745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:48.144754 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:08:48.144791 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:48.144793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:48.144809 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:48.144916 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:08:48.145148 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.146682 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.148509 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.149276 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.149349 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:48.149421 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:48.150230 (+ 809us) server_negotiation.cc:299] Negotiation successful
0925 13:08:48.150268 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":985,"thread_start_us":73,"threads_started":1}
I20260925 13:08:48.151226 12458 tablet_service.cc:1467] Processing CreateTablet for tablet 0189e9ecb2de4b9487151761ba980e11 (DEFAULT_TABLE table=test-table [id=263842736b14402387c781a92b720718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:08:48.151569 12458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0189e9ecb2de4b9487151761ba980e11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:08:48.155565 12554 tablet_bootstrap.cc:492] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08: Bootstrap starting.
I20260925 13:08:48.155841 12548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:48.143947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:40301 (local address 127.0.0.1:41602)
0925 13:08:48.146198 (+ 2251us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:48.146207 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:48.146250 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:48.151051 (+ 4801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:48.151052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:48.151055 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:08:48.151096 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:48.151098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.153195 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.153196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:48.154463 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:48.154465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.155578 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.155579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:48.155649 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:48.155653 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:08:48.155719 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 13:08:48.155760 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2236,"thread_start_us":65,"threads_started":1}
I20260925 13:08:48.156790 12554 tablet_bootstrap.cc:654] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08: Neither blocks nor log segments found. Creating new log.
I20260925 13:08:48.157222 12554 log.cc:824] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08: Log is configured to *not* fsync() on all Append() calls
I20260925 13:08:48.158039 12554 tablet_bootstrap.cc:492] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08: No bootstrap required, opened a new log
I20260925 13:08:48.158105 12554 ts_tablet_manager.cc:1397] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 13:08:48.160445 12554 raft_consensus.cc:348] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "496f9417bbb14a6d851ccf5d67f88f08" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37965 } } peers { permanent_uuid: "1ec51e6cf69947479a79e2c91c12c995" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40301 } } peers { permanent_uuid: "f0fe2efdf1ac42adba83ecdce3fe2b18" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46411 } }
I20260925 13:08:48.160614 12554 raft_consensus.cc:374] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:08:48.160655 12554 raft_consensus.cc:729] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 496f9417bbb14a6d851ccf5d67f88f08, State: Initialized, Role: FOLLOWER
I20260925 13:08:48.160759 12554 consensus_queue.cc:260] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "496f9417bbb14a6d851ccf5d67f88f08" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37965 } } peers { permanent_uuid: "1ec51e6cf69947479a79e2c91c12c995" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40301 } } peers { permanent_uuid: "f0fe2efdf1ac42adba83ecdce3fe2b18" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46411 } }
I20260925 13:08:48.161041 12554 ts_tablet_manager.cc:1428] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260925 13:08:48.161376 12535 heartbeater.cc:503] Master 127.0.206.126:43949 was elected leader, sending a full tablet report...
I20260925 13:08:48.165462 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:48.144050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41602 (local address 127.0.206.65:40301)
0925 13:08:48.150798 (+ 6748us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:48.150803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:48.150816 (+ 13us) server_negotiation.cc:407] Connection header received
0925 13:08:48.150855 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:48.150858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:48.150876 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:48.150974 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:08:48.151347 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.152573 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.154572 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.155403 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.155480 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:48.155538 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:48.165325 (+ 9787us) server_negotiation.cc:299] Negotiation successful
0925 13:08:48.165383 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6707,"thread_start_us":63,"threads_started":1}
I20260925 13:08:48.166076 12195 tablet_service.cc:1467] Processing CreateTablet for tablet 0189e9ecb2de4b9487151761ba980e11 (DEFAULT_TABLE table=test-table [id=263842736b14402387c781a92b720718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:08:48.166402 12195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0189e9ecb2de4b9487151761ba980e11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:08:48.170307 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:48.143502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:46411 (local address 127.0.0.1:53394)
0925 13:08:48.144573 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:48.144582 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:48.144615 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:48.153052 (+ 8437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:48.153053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:48.153056 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:08:48.153096 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:48.153098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.165280 (+ 12182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.165281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:48.166400 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:48.166404 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.170063 (+ 3659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.170063 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:48.170131 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:48.170136 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 13:08:48.170184 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 13:08:48.170229 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1058,"thread_start_us":71,"threads_started":1}
I20260925 13:08:48.170749 12550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:48.143630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53394 (local address 127.0.206.66:46411)
0925 13:08:48.152752 (+ 9122us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:48.152757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:48.152766 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:08:48.152867 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:48.152870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:48.152886 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:48.152968 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:08:48.156021 (+ 3053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.157225 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.167485 (+ 10260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.168268 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.170483 (+ 2215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:48.170573 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:48.170600 (+ 27us) server_negotiation.cc:299] Negotiation successful
0925 13:08:48.170644 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9080,"thread_start_us":104,"threads_started":1}
I20260925 13:08:48.171428 12334 tablet_service.cc:1467] Processing CreateTablet for tablet 0189e9ecb2de4b9487151761ba980e11 (DEFAULT_TABLE table=test-table [id=263842736b14402387c781a92b720718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:08:48.171784 12334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0189e9ecb2de4b9487151761ba980e11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:08:48.172818 12557 tablet_bootstrap.cc:492] T 0189e9ecb2de4b9487151761ba980e11 P 1ec51e6cf69947479a79e2c91c12c995: Bootstrap starting.
I20260925 13:08:48.173419 12557 tablet_bootstrap.cc:654] T 0189e9ecb2de4b9487151761ba980e11 P 1ec51e6cf69947479a79e2c91c12c995: Neither blocks nor log segments found. Creating new log.
I20260925 13:08:48.173807 12557 log.cc:824] T 0189e9ecb2de4b9487151761ba980e11 P 1ec51e6cf69947479a79e2c91c12c995: Log is configured to *not* fsync() on all Append() calls
I20260925 13:08:48.175722 12559 tablet_bootstrap.cc:492] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18: Bootstrap starting.
I20260925 13:08:48.176733 12559 tablet_bootstrap.cc:654] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18: Neither blocks nor log segments found. Creating new log.
I20260925 13:08:48.177028 12559 log.cc:824] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18: Log is configured to *not* fsync() on all Append() calls
I20260925 13:08:48.177749 12559 tablet_bootstrap.cc:492] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18: No bootstrap required, opened a new log
I20260925 13:08:48.177821 12559 ts_tablet_manager.cc:1397] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:08:48.179795 12557 tablet_bootstrap.cc:492] T 0189e9ecb2de4b9487151761ba980e11 P 1ec51e6cf69947479a79e2c91c12c995: No bootstrap required, opened a new log
I20260925 13:08:48.179872 12557 ts_tablet_manager.cc:1397] T 0189e9ecb2de4b9487151761ba980e11 P 1ec51e6cf69947479a79e2c91c12c995: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260925 13:08:48.180219 12559 raft_consensus.cc:348] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "496f9417bbb14a6d851ccf5d67f88f08" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37965 } } peers { permanent_uuid: "1ec51e6cf69947479a79e2c91c12c995" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40301 } } peers { permanent_uuid: "f0fe2efdf1ac42adba83ecdce3fe2b18" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46411 } }
I20260925 13:08:48.180415 12559 raft_consensus.cc:374] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:08:48.180454 12559 raft_consensus.cc:729] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0fe2efdf1ac42adba83ecdce3fe2b18, State: Initialized, Role: FOLLOWER
I20260925 13:08:48.180591 12559 consensus_queue.cc:260] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "496f9417bbb14a6d851ccf5d67f88f08" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37965 } } peers { permanent_uuid: "1ec51e6cf69947479a79e2c91c12c995" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40301 } } peers { permanent_uuid: "f0fe2efdf1ac42adba83ecdce3fe2b18" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46411 } }
I20260925 13:08:48.180859 12559 ts_tablet_manager.cc:1428] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 13:08:48.180977 12401 heartbeater.cc:503] Master 127.0.206.126:43949 was elected leader, sending a full tablet report...
I20260925 13:08:48.185319 12557 raft_consensus.cc:348] T 0189e9ecb2de4b9487151761ba980e11 P 1ec51e6cf69947479a79e2c91c12c995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "496f9417bbb14a6d851ccf5d67f88f08" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37965 } } peers { permanent_uuid: "1ec51e6cf69947479a79e2c91c12c995" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40301 } } peers { permanent_uuid: "f0fe2efdf1ac42adba83ecdce3fe2b18" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46411 } }
I20260925 13:08:48.185483 12557 raft_consensus.cc:374] T 0189e9ecb2de4b9487151761ba980e11 P 1ec51e6cf69947479a79e2c91c12c995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:08:48.185521 12557 raft_consensus.cc:729] T 0189e9ecb2de4b9487151761ba980e11 P 1ec51e6cf69947479a79e2c91c12c995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ec51e6cf69947479a79e2c91c12c995, State: Initialized, Role: FOLLOWER
I20260925 13:08:48.185665 12557 consensus_queue.cc:260] T 0189e9ecb2de4b9487151761ba980e11 P 1ec51e6cf69947479a79e2c91c12c995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "496f9417bbb14a6d851ccf5d67f88f08" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37965 } } peers { permanent_uuid: "1ec51e6cf69947479a79e2c91c12c995" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40301 } } peers { permanent_uuid: "f0fe2efdf1ac42adba83ecdce3fe2b18" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46411 } }
I20260925 13:08:48.186048 12557 ts_tablet_manager.cc:1428] T 0189e9ecb2de4b9487151761ba980e11 P 1ec51e6cf69947479a79e2c91c12c995: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260925 13:08:48.186172 12261 heartbeater.cc:503] Master 127.0.206.126:43949 was elected leader, sending a full tablet report...
I20260925 13:08:48.186822 12561 raft_consensus.cc:482] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:08:48.186918 12561 raft_consensus.cc:504] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "496f9417bbb14a6d851ccf5d67f88f08" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37965 } } peers { permanent_uuid: "1ec51e6cf69947479a79e2c91c12c995" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40301 } } peers { permanent_uuid: "f0fe2efdf1ac42adba83ecdce3fe2b18" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46411 } }
I20260925 13:08:48.187749 12561 leader_election.cc:288] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 496f9417bbb14a6d851ccf5d67f88f08 (127.0.206.67:37965), 1ec51e6cf69947479a79e2c91c12c995 (127.0.206.65:40301)
I20260925 13:08:48.190917 12403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:48.187407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:37965 (local address 127.0.206.66:55079)
0925 13:08:48.187481 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:48.187494 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:48.187555 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:48.187834 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:48.187836 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:48.187837 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:08:48.187899 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:48.187901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.188782 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.188782 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:48.189883 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:48.189885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.190589 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.190590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:48.190662 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:48.190666 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:08:48.190801 (+ 135us) client_negotiation.cc:240] Negotiation successful
0925 13:08:48.190852 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260925 13:08:48.191394 12551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:48.187470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:55079 (local address 127.0.206.67:37965)
0925 13:08:48.187514 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:48.187516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:48.187606 (+ 90us) server_negotiation.cc:407] Connection header received
0925 13:08:48.187651 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:48.187652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:48.187663 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:48.187705 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:08:48.187970 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.188678 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.189978 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.190488 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.190551 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:48.190560 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:48.191295 (+ 735us) server_negotiation.cc:299] Negotiation successful
0925 13:08:48.191336 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 13:08:48.191640 12484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0189e9ecb2de4b9487151761ba980e11" candidate_uuid: "f0fe2efdf1ac42adba83ecdce3fe2b18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496f9417bbb14a6d851ccf5d67f88f08" is_pre_election: true
I20260925 13:08:48.191762 12484 raft_consensus.cc:2454] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0fe2efdf1ac42adba83ecdce3fe2b18 in term 0.
I20260925 13:08:48.191982 12288 leader_election.cc:302] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496f9417bbb14a6d851ccf5d67f88f08, f0fe2efdf1ac42adba83ecdce3fe2b18; no voters:
I20260925 13:08:48.193099 12561 raft_consensus.cc:2781] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:08:48.193171 12561 raft_consensus.cc:482] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:08:48.193192 12561 raft_consensus.cc:3037] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:08:48.194054 12561 raft_consensus.cc:504] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "496f9417bbb14a6d851ccf5d67f88f08" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37965 } } peers { permanent_uuid: "1ec51e6cf69947479a79e2c91c12c995" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40301 } } peers { permanent_uuid: "f0fe2efdf1ac42adba83ecdce3fe2b18" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46411 } }
I20260925 13:08:48.194548 12561 leader_election.cc:288] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [CANDIDATE]: Term 1 election: Requested vote from peers 496f9417bbb14a6d851ccf5d67f88f08 (127.0.206.67:37965), 1ec51e6cf69947479a79e2c91c12c995 (127.0.206.65:40301)
I20260925 13:08:48.194773 12484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0189e9ecb2de4b9487151761ba980e11" candidate_uuid: "f0fe2efdf1ac42adba83ecdce3fe2b18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "496f9417bbb14a6d851ccf5d67f88f08"
I20260925 13:08:48.194844 12484 raft_consensus.cc:3037] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:08:48.195603 12484 raft_consensus.cc:2454] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0fe2efdf1ac42adba83ecdce3fe2b18 in term 1.
I20260925 13:08:48.195783 12288 leader_election.cc:302] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 496f9417bbb14a6d851ccf5d67f88f08, f0fe2efdf1ac42adba83ecdce3fe2b18; no voters:
I20260925 13:08:48.200680 12403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:48.187563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:40301 (local address 127.0.206.66:50551)
0925 13:08:48.190989 (+ 3426us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:48.191002 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:08:48.191044 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:08:48.192274 (+ 1230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:08:48.192275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:08:48.192276 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:08:48.192305 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:48.192306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.195898 (+ 3592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.195899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:48.196884 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:08:48.196887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.200421 (+ 3534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.200422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:08:48.200492 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:48.200495 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:08:48.200547 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 13:08:48.200580 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3412,"thread_start_us":109,"threads_started":1}
I20260925 13:08:48.201128 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:48.187769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:50551 (local address 127.0.206.65:40301)
0925 13:08:48.187805 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:48.187807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:48.191156 (+ 3349us) server_negotiation.cc:407] Connection header received
0925 13:08:48.191180 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:48.191181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:48.191190 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:48.191231 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:08:48.192379 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.193018 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.196981 (+ 3963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.197540 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.197607 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:48.197617 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:48.201018 (+ 3401us) server_negotiation.cc:299] Negotiation successful
0925 13:08:48.201067 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 13:08:48.201391 12215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0189e9ecb2de4b9487151761ba980e11" candidate_uuid: "f0fe2efdf1ac42adba83ecdce3fe2b18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ec51e6cf69947479a79e2c91c12c995" is_pre_election: true
I20260925 13:08:48.201536 12215 raft_consensus.cc:2454] T 0189e9ecb2de4b9487151761ba980e11 P 1ec51e6cf69947479a79e2c91c12c995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0fe2efdf1ac42adba83ecdce3fe2b18 in term 0.
I20260925 13:08:48.201637 12214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0189e9ecb2de4b9487151761ba980e11" candidate_uuid: "f0fe2efdf1ac42adba83ecdce3fe2b18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ec51e6cf69947479a79e2c91c12c995"
I20260925 13:08:48.201687 12214 raft_consensus.cc:3037] T 0189e9ecb2de4b9487151761ba980e11 P 1ec51e6cf69947479a79e2c91c12c995 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:08:48.202677 12214 raft_consensus.cc:2454] T 0189e9ecb2de4b9487151761ba980e11 P 1ec51e6cf69947479a79e2c91c12c995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0fe2efdf1ac42adba83ecdce3fe2b18 in term 1.
I20260925 13:08:48.203174 12561 raft_consensus.cc:2781] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:08:48.203327 12561 raft_consensus.cc:686] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [term 1 LEADER]: Becoming Leader. State: Replica: f0fe2efdf1ac42adba83ecdce3fe2b18, State: Running, Role: LEADER
I20260925 13:08:48.203411 12561 consensus_queue.cc:237] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "496f9417bbb14a6d851ccf5d67f88f08" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37965 } } peers { permanent_uuid: "1ec51e6cf69947479a79e2c91c12c995" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40301 } } peers { permanent_uuid: "f0fe2efdf1ac42adba83ecdce3fe2b18" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46411 } }
I20260925 13:08:48.204312 12050 catalog_manager.cc:5495] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0fe2efdf1ac42adba83ecdce3fe2b18 (127.0.206.66). New cstate: current_term: 1 leader_uuid: "f0fe2efdf1ac42adba83ecdce3fe2b18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "496f9417bbb14a6d851ccf5d67f88f08" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ec51e6cf69947479a79e2c91c12c995" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0fe2efdf1ac42adba83ecdce3fe2b18" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46411 } health_report { overall_health: HEALTHY } } }
I20260925 13:08:48.211856 12550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:48.208525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53410 (local address 127.0.206.66:46411)
0925 13:08:48.208560 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 13:08:48.208562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:08:48.208568 (+ 6us) server_negotiation.cc:407] Connection header received
0925 13:08:48.208597 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:08:48.208597 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:08:48.208615 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:08:48.208677 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:08:48.208872 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.209606 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.210791 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:08:48.211086 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:08:48.211161 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:08:48.211368 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:08:48.211484 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:08:48.211551 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 13:08:48.211753 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 13:08:48.211772 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:08:48.213766 12214 raft_consensus.cc:1261] T 0189e9ecb2de4b9487151761ba980e11 P 1ec51e6cf69947479a79e2c91c12c995 [term 1 FOLLOWER]: Refusing update from remote peer f0fe2efdf1ac42adba83ecdce3fe2b18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:08:48.214103 12564 consensus_queue.cc:1035] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ec51e6cf69947479a79e2c91c12c995" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:08:48.215775 12484 raft_consensus.cc:1261] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08 [term 1 FOLLOWER]: Refusing update from remote peer f0fe2efdf1ac42adba83ecdce3fe2b18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:08:48.216112 12561 consensus_queue.cc:1035] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "496f9417bbb14a6d851ccf5d67f88f08" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:08:48.220311 12567 mvcc.cc:205] Tried to move back new op lower bound from 7333239718761037824 to 7333239718722134016. Current Snapshot: MvccSnapshot[applied={T|T < 7333239718761037824}]
I20260925 13:08:48.220449 12570 mvcc.cc:205] Tried to move back new op lower bound from 7333239718761037824 to 7333239718722134016. Current Snapshot: MvccSnapshot[applied={T|T < 7333239718761037824}]
I20260925 13:08:48.222584 12568 mvcc.cc:205] Tried to move back new op lower bound from 7333239718761037824 to 7333239718722134016. Current Snapshot: MvccSnapshot[applied={T|T < 7333239718761037824}]
I20260925 13:08:48.225194 12050 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:08:48.225274 12050 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:08:48.225973 12050 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b418cdd7906d49d8bde8b5b1a68cbed6: Sending DeleteTablet for 3 replicas of tablet 0189e9ecb2de4b9487151761ba980e11
I20260925 13:08:48.226387 12458 tablet_service.cc:1514] Processing DeleteTablet for tablet 0189e9ecb2de4b9487151761ba980e11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:08:48 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:42466
I20260925 13:08:48.226637 12334 tablet_service.cc:1514] Processing DeleteTablet for tablet 0189e9ecb2de4b9487151761ba980e11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:08:48 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:53394
I20260925 13:08:48.227051 12195 tablet_service.cc:1514] Processing DeleteTablet for tablet 0189e9ecb2de4b9487151761ba980e11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:08:48 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:41602
I20260925 13:08:48.227553 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 12132
I20260925 13:08:48.231513 12576 tablet_replica.cc:330] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08: stopping tablet replica
I20260925 13:08:48.231647 12577 tablet_replica.cc:330] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18: stopping tablet replica
I20260925 13:08:48.231812 12577 raft_consensus.cc:2229] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:08:48.231961 12576 raft_consensus.cc:2229] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:08:48.232131 12577 raft_consensus.cc:2258] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:08:48.232291 12576 raft_consensus.cc:2258] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:08:48.232985 12576 ts_tablet_manager.cc:1905] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:08:48.233317 12577 ts_tablet_manager.cc:1905] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:08:48.237038 12576 ts_tablet_manager.cc:1918] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:08:48.246795 12576 log.cc:1196] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/0189e9ecb2de4b9487151761ba980e11
I20260925 13:08:48.247038 12577 ts_tablet_manager.cc:1918] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:08:48.247440 12576 ts_tablet_manager.cc:1939] T 0189e9ecb2de4b9487151761ba980e11 P 496f9417bbb14a6d851ccf5d67f88f08: Deleting consensus metadata
I20260925 13:08:48.247550 12577 log.cc:1196] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/0189e9ecb2de4b9487151761ba980e11
I20260925 13:08:48.247871 12577 ts_tablet_manager.cc:1939] T 0189e9ecb2de4b9487151761ba980e11 P f0fe2efdf1ac42adba83ecdce3fe2b18: Deleting consensus metadata
I20260925 13:08:48.248461 12035 catalog_manager.cc:4841] TS 496f9417bbb14a6d851ccf5d67f88f08 (127.0.206.67:37965): tablet 0189e9ecb2de4b9487151761ba980e11 (table test-table [id=263842736b14402387c781a92b720718]) successfully deleted
I20260925 13:08:48.248742 12035 catalog_manager.cc:4841] TS f0fe2efdf1ac42adba83ecdce3fe2b18 (127.0.206.66:46411): tablet 0189e9ecb2de4b9487151761ba980e11 (table test-table [id=263842736b14402387c781a92b720718]) successfully deleted
W20260925 13:08:48.249100 12037 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:40301 (error 108)
I20260925 13:08:48.251693 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:08:48.251544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:40301 (local address 127.0.0.1:41608)
0925 13:08:48.251608 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 13:08:48.251658 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:40301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260925 13:08:48.251817 12037 catalog_manager.cc:4584] TS 1ec51e6cf69947479a79e2c91c12c995 (127.0.206.65:40301): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0189e9ecb2de4b9487151761ba980e11: Network error: Client connection negotiation failed: client connection to 127.0.206.65:40301: connect: Connection refused (error 111)
I20260925 13:08:48.251935 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 12269
I20260925 13:08:48.258795 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 12406
I20260925 13:08:48.265607 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 12018
I20260925 13:08:48.272351 825 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 13:08:48.606950 825 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 13:08:48.607039 825 mini_ranger.cc:66] Stopping Ranger...
I20260925 13:08:48.951260 825 mini_ranger.cc:68] Stopped Ranger
2026-09-25 13:08:48.951 UTC [11604] LOG: received smart shutdown request
2026-09-25 13:08:48.954 UTC [11604] LOG: background worker "logical replication launcher" (PID 11613) exited with exit code 1
2026-09-25 13:08:48.955 UTC [11608] LOG: shutting down
2026-09-25 13:08:49.138 UTC [11604] LOG: database system is shut down
2026-09-25T13:08:49Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (43652 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:08:49 dist-test-slave-chkf krb5kdc[12588](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:08:49 dist-test-slave-chkf krb5kdc[12588](info): set up 2 sockets
Sep 25 13:08:49 dist-test-slave-chkf krb5kdc[12588](info): commencing operation
krb5kdc: starting...
W20260925 13:08:50.274668 825 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:08:50 dist-test-slave-chkf krb5kdc[12588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:08:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:08:50Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.206.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.206.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/rangeradmin_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/rangeradmin_127.0.206.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.206.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.206.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/rangerlookup_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/rangerlookup_127.0.206.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.206.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.206.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/HTTP_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/HTTP_127.0.206.84@KRBTEST.COM.keytab.
I20260925 13:08:50.440769 825 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskG_Ts_H/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/postgres -l logfile start
2026-09-25 13:08:52.913 UTC [12618] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 13:08:52.913 UTC [12618] LOG: listening on IPv4 address "127.0.206.84", port 49139
2026-09-25 13:08:52.916 UTC [12618] LOG: listening on Unix socket "/tmp/.s.PGSQL.49139"
2026-09-25 13:08:52.950 UTC [12621] LOG: database system was shut down at 2026-09-25 13:08:51 UTC
2026-09-25 13:08:52.955 UTC [12618] LOG: database system is ready to accept connections
I20260925 13:08:53.894614 825 mini_postgres.cc:96] Postgres bound to 49139
2026-09-25 13:08:53.900 UTC [12629] FATAL: database "slave" does not exist
127.0.206.84:49139 - accepting connections
I20260925 13:08:53.900985 825 mini_ranger.cc:161] Starting Ranger...
I20260925 13:08:53.914700 825 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 13:08:54.356982 825 mini_ranger.cc:87] Created ranger Postgres database
I20260925 13:08:54.357070 825 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-admin
2026-09-25 13:08:54,637 [I] DB FLAVOR :POSTGRES
2026-09-25 13:08:54,637 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 13:08:54,638 [I] Checking connection..
2026-09-25 13:08:54,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 13:08:54,895 [I] Checking connection passed.
2026-09-25 13:08:54,895 [I] --------- Verifying version history table ---------
2026-09-25 13:08:54,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 13:08:55,147 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 13:08:55,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 13:08:55,383 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 13:08:55,383 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 13:08:55,383 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 13:08:55.607 UTC [12693] WARNING: there is no transaction in progress
2026-09-25 13:08:55,613 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 13:08:55,613 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 13:08:55,845 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 13:08:55,845 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 13:08:55,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 13:08:56,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 13:08:56,299 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-chkf.c.gcp-upstream.internal','N') ;"
2026-09-25 13:08:56,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 13:08:56,799 [I] Table x_portal_user does not exist in database ranger
2026-09-25 13:08:56,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 13:08:57,045 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 13:08:57,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 13:08:57,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 13:08:57,525 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 13:08:57,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 13:08:58.403 UTC [12801] WARNING: there is no transaction in progress
2026-09-25 13:08:58.418 UTC [12801] WARNING: there is no transaction in progress
2026-09-25 13:08:58.432 UTC [12801] WARNING: there is no transaction in progress
2026-09-25 13:08:58.446 UTC [12801] WARNING: there is no transaction in progress
2026-09-25 13:08:58.463 UTC [12801] WARNING: there is no transaction in progress
2026-09-25 13:08:58.481 UTC [12801] WARNING: there is no transaction in progress
2026-09-25 13:08:58.545 UTC [12801] WARNING: there is no transaction in progress
2026-09-25 13:08:58.551 UTC [12801] WARNING: there is no transaction in progress
2026-09-25 13:08:58.564 UTC [12801] WARNING: there is no transaction in progress
2026-09-25 13:08:58.573 UTC [12801] WARNING: there is no transaction in progress
2026-09-25 13:08:58.582 UTC [12801] WARNING: there is no transaction in progress
2026-09-25 13:08:58.595 UTC [12801] WARNING: there is no transaction in progress
2026-09-25 13:08:58.603 UTC [12801] WARNING: there is no transaction in progress
2026-09-25 13:08:58.609 UTC [12801] WARNING: there is no transaction in progress
2026-09-25 13:08:58.615 UTC [12801] WARNING: there is no transaction in progress
2026-09-25 13:08:58.621 UTC [12801] WARNING: there is no transaction in progress
2026-09-25 13:08:59,212 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 13:08:59,213 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 13:08:59,435 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 13:08:59,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 13:08:59,677 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 13:08:59,677 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 13:08:59,927 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 13:08:59,927 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 13:09:00,152 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 13:09:00,395 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-chkf.c.gcp-upstream.internal';"
2026-09-25 13:09:00,635 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 13:09:00,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 13:09:00,872 [I] DB_PATCHES have already been applied
I20260925 13:09:00.878002 825 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-admin:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskG_Ts_H/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 13:09:00.878060 825 mini_ranger.cc:193] Using host: 127.0.206.84
I20260925 13:09:00.881143 825 mini_ranger.cc:227] Ranger admin URL: http://127.0.206.84:36773
Sep 25, 2026 1:09:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 1:09:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 1:09:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 1:09:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 1:09:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 1:09:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 1:09:01 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 13:09:01.984803 825 mini_ranger.cc:160] Time spent starting Ranger: real 8.084s user 0.000s sys 0.008s
Sep 25, 2026 1:09:20 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 1:09:20 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 25, 2026 1:09:20 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 1:09:20 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 1:09:20 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 1:09:20 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 1:09:20 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 1:09:20 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 1:09:20 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 1:09:20 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 1:09:20 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 1:09:20 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 1:09:20 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 1:09:20 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 1:09:20 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 1:09:20 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 1:09:20 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 1:09:21 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 13:09:21.854540 825 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.206.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.206.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/rangerkms_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/rangerkms_127.0.206.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.206.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.206.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.206.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/HTTP_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.206.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/HTTP_127.0.206.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 13:09:21 dist-test-slave-chkf krb5kdc[12588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341761, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 13:09:21.933676 825 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 13:09:21.942131 825 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 13:09:22.782656 825 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 13:09:22.782806 825 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-kms
I20260925 13:09:22.782832 825 mini_ranger_kms.cc:227] Using postgres at 127.0.206.84:49139
2026-09-25 13:09:22,948 [I] DB FLAVOR :POSTGRES
2026-09-25 13:09:22,948 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 13:09:22,948 [I] Checking connection
2026-09-25 13:09:22,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 13:09:23,168 [I] connection success
2026-09-25 13:09:23,169 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 13:09:23,169 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 13:09:23,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 13:09:23,403 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 13:09:23,403 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 13:09:23,404 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 13:09:23,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:49139/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 13:09:23,660 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 13:09:24.073037 825 mini_ranger_kms.cc:310] Created kms service
I20260925 13:09:24.248565 825 mini_ranger_kms.cc:326] Created kudu user
I20260925 13:09:24.288679 825 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 13:09:24 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341761, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260925 13:09:24.668995 825 mini_ranger_kms.cc:384] Added ranger policy
I20260925 13:09:24.669101 825 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-kms:/tmp/dist-test-taskG_Ts_H/build/release/bin/postgresql.jar:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskG_Ts_H/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 13:09:24.669121 825 mini_ranger_kms.cc:240] Using host: 127.0.206.84
I20260925 13:09:24.672480 825 mini_ranger_kms.cc:276] Ranger KMS PID: 12981
I20260925 13:09:24.672552 825 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.206.84:54173
Sep 25, 2026 1:09:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 1:09:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 1:09:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 1:09:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 1:09:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 1:09:26 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 25 13:09:26 dist-test-slave-chkf krb5kdc[12588](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790341766, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.206.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:26.914983 825 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.981s user 0.005s sys 0.144s
I20260925 13:09:26.915088 825 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 13:09:26.915139 825 mini_ranger_kms.cc:401] 127.0.206.84:54173/kms/v1/keys
Sep 25 13:09:27 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341766, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.206.84@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
Sep 25 13:09:27 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341766, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.206.84@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
Sep 25 13:09:29 dist-test-slave-chkf krb5kdc[12588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:09:29.737910 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:39597
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:39597
--ranger_config_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:54173/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:29.816884 13032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:29.817106 13032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:29.817142 13032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:29.818272 13032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:09:29.818336 13032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:29.818362 13032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:29.818392 13032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:09:29.818425 13032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:09:29.820007 13032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.206.84:54173/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:39597
--ranger_config_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:39597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:29.820274 13032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:29.820487 13032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:29.825390 13037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:29.825409 13038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:29.825420 13040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:29.825418 13032 server_base.cc:1034] running on GCE node
I20260925 13:09:29.825888 13032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:29.826319 13032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:29.827556 13032 hybrid_clock.cc:648] HybridClock initialized: now 1790341769827528 us; error 45 us; skew 500 ppm
Sep 25 13:09:29 dist-test-slave-chkf krb5kdc[12588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341769, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:29.829941 13032 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:09:29.830375 13032 webserver.cc:460] Webserver started at http://127.0.206.126:46809/ using document root <none> and password file <none>
I20260925 13:09:29.830547 13032 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:29.830582 13032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:29.830667 13032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 13:09:29 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341769, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260925 13:09:29.885317 13032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "6f2abf44e39644568f29d76b85f17ee8"
format_stamp: "Formatted at 2026-09-25 13:09:29 on dist-test-slave-chkf"
server_key: "293282cf75acc529eda44a7e2959ccb4"
server_key_iv: "a5e61b77192e04dbf8d797ca5f8aa2c4"
server_key_version: "kuduclusterkey@0"
I20260925 13:09:29.885797 13032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "6f2abf44e39644568f29d76b85f17ee8"
format_stamp: "Formatted at 2026-09-25 13:09:29 on dist-test-slave-chkf"
server_key: "293282cf75acc529eda44a7e2959ccb4"
server_key_iv: "a5e61b77192e04dbf8d797ca5f8aa2c4"
server_key_version: "kuduclusterkey@0"
I20260925 13:09:29.887605 13032 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260925 13:09:29.898583 13047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:29.898808 13032 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:09:29.898909 13032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "6f2abf44e39644568f29d76b85f17ee8"
format_stamp: "Formatted at 2026-09-25 13:09:29 on dist-test-slave-chkf"
server_key: "293282cf75acc529eda44a7e2959ccb4"
server_key_iv: "a5e61b77192e04dbf8d797ca5f8aa2c4"
server_key_version: "kuduclusterkey@0"
I20260925 13:09:29.899016 13032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:29.917596 13032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:29.919754 13032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:29.919920 13032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:29.924504 13032 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:39597
I20260925 13:09:29.924608 13099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:39597 every 8 connection(s)
I20260925 13:09:29.924935 13032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:09:29.925585 13100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:29.927831 13100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8: Bootstrap starting.
I20260925 13:09:29.928395 13100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:29.928691 13100 log.cc:824] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:29.929354 13100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8: No bootstrap required, opened a new log
I20260925 13:09:29.931373 13100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2abf44e39644568f29d76b85f17ee8" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39597 } }
I20260925 13:09:29.931512 13100 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:29.931535 13100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f2abf44e39644568f29d76b85f17ee8, State: Initialized, Role: FOLLOWER
I20260925 13:09:29.931716 13100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2abf44e39644568f29d76b85f17ee8" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39597 } }
I20260925 13:09:29.931779 13100 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:09:29.931819 13100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:09:29.931867 13100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:29.932374 13100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2abf44e39644568f29d76b85f17ee8" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39597 } }
I20260925 13:09:29.932457 13100 leader_election.cc:302] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f2abf44e39644568f29d76b85f17ee8; no voters:
I20260925 13:09:29.932647 13100 leader_election.cc:288] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:09:29.932687 13103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:09:29.932835 13103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [term 1 LEADER]: Becoming Leader. State: Replica: 6f2abf44e39644568f29d76b85f17ee8, State: Running, Role: LEADER
I20260925 13:09:29.932904 13100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:09:29.932972 13103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2abf44e39644568f29d76b85f17ee8" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39597 } }
I20260925 13:09:29.933531 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 13032
I20260925 13:09:29.933645 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:09:29.933866 825 external_mini_cluster.cc:1428] Setting key 0318a8e55f86ef03c78e60540373e69e
I20260925 13:09:29.934007 13104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f2abf44e39644568f29d76b85f17ee8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2abf44e39644568f29d76b85f17ee8" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39597 } } }
I20260925 13:09:29.934024 13105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f2abf44e39644568f29d76b85f17ee8. Latest consensus state: current_term: 1 leader_uuid: "6f2abf44e39644568f29d76b85f17ee8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2abf44e39644568f29d76b85f17ee8" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39597 } } }
I20260925 13:09:29.934234 13105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [sys.catalog]: This master's current role is: LEADER
I20260925 13:09:29.934453 13104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [sys.catalog]: This master's current role is: LEADER
I20260925 13:09:29.934607 13114 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:09:29.935137 13114 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:09:29.935395 13100 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 13:09:29.943308 13114 catalog_manager.cc:1347] Generated new cluster ID: 174d07d807aa4e9ab20a9b5835a59ff9
I20260925 13:09:29.943396 13114 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 13:09:29 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:29.948565 13115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:29.934330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53406 (local address 127.0.206.126:39597)
0925 13:09:29.934492 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:29.934501 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:29.934572 (+ 71us) server_negotiation.cc:407] Connection header received
0925 13:09:29.934745 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:29.934754 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:29.934823 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:29.935010 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:29.935761 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:29.939327 (+ 3566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:29.940415 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:29.941112 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:29.941243 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:29.944350 (+ 3107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:29.944354 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:29.944355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:29.944367 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:29.946461 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:29.946903 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:29.946907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:29.946907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:29.947013 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:29.947230 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:29.947231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:29.947231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:29.947564 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:29.947717 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:29.948212 (+ 495us) server_negotiation.cc:299] Negotiation successful
0925 13:09:29.948338 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260925 13:09:29.957231 13114 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:09:29.957963 13114 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:09:29.964303 13114 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8: Generated new TSK 0
I20260925 13:09:29.964648 13114 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260925 13:09:30.617900 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:54173/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39597
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:30.726545 13145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:30.726790 13145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:30.726830 13145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:30.727998 13145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:30.728063 13145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:30.728107 13145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:09:30.729617 13145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.206.84:54173/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:30.729882 13145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:30.730130 13145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:30.735639 13151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:30.735842 13153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:30.735607 13150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:30.736142 13145 server_base.cc:1034] running on GCE node
I20260925 13:09:30.736356 13145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:30.736629 13145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:30.737775 13145 hybrid_clock.cc:648] HybridClock initialized: now 1790341770737758 us; error 39 us; skew 500 ppm
Sep 25 13:09:30 dist-test-slave-chkf krb5kdc[12588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341770, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:30.740384 13145 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:09:30.740869 13145 webserver.cc:460] Webserver started at http://127.0.206.65:37899/ using document root <none> and password file <none>
I20260925 13:09:30.741119 13145 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:30.741192 13145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:30.741340 13145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 13:09:30 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341770, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260925 13:09:30.754522 13145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "cf55337cdcad4caa9663ba2dd6e31b0a"
format_stamp: "Formatted at 2026-09-25 13:09:30 on dist-test-slave-chkf"
server_key: "d4183f1d38f1b254ba76c826971ea228"
server_key_iv: "eb45dc1d017ee20e3d1bf3b114c42fd7"
server_key_version: "kuduclusterkey@0"
I20260925 13:09:30.754952 13145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "cf55337cdcad4caa9663ba2dd6e31b0a"
format_stamp: "Formatted at 2026-09-25 13:09:30 on dist-test-slave-chkf"
server_key: "d4183f1d38f1b254ba76c826971ea228"
server_key_iv: "eb45dc1d017ee20e3d1bf3b114c42fd7"
server_key_version: "kuduclusterkey@0"
I20260925 13:09:30.758810 13145 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260925 13:09:30.771440 13160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:30.771654 13145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 13:09:30.771791 13145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "cf55337cdcad4caa9663ba2dd6e31b0a"
format_stamp: "Formatted at 2026-09-25 13:09:30 on dist-test-slave-chkf"
server_key: "d4183f1d38f1b254ba76c826971ea228"
server_key_iv: "eb45dc1d017ee20e3d1bf3b114c42fd7"
server_key_version: "kuduclusterkey@0"
I20260925 13:09:30.771893 13145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:30.794256 13145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:30.796437 13145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:30.796602 13145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:30.796885 13145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:09:30.797276 13145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:09:30.797329 13145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:30.797396 13145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:09:30.797441 13145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:30.804601 13145 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:40731
I20260925 13:09:30.804700 13275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:40731 every 8 connection(s)
I20260925 13:09:30.805014 13145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:09:30.807192 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 13145
I20260925 13:09:30.807296 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:09:30.807508 825 external_mini_cluster.cc:1428] Setting key fe32153712db987e905ce20cbd348802
Sep 25 13:09:30 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341770, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:30.815841 13279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:30.805768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:39597 (local address 127.0.206.65:43723)
0925 13:09:30.805993 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:30.806023 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:30.806538 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:30.806846 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:30.806850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:30.807036 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:30.807260 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:30.807265 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:30.808315 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:30.808316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:30.809594 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:30.809597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:30.810026 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:30.810028 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:30.810160 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:30.810736 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:30.810746 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:30.812046 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:30.814211 (+ 2165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:30.814217 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:30.814219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:30.814503 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:30.814745 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:30.814746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:30.814746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:30.814879 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:30.815210 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:30.815216 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:30.815370 (+ 154us) client_negotiation.cc:769] Sending connection context
0925 13:09:30.815455 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 13:09:30.815605 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":75,"threads_started":1}
I20260925 13:09:30.816704 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:30.805765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:43723 (local address 127.0.206.126:39597)
0925 13:09:30.805932 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:30.805934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:30.806120 (+ 186us) server_negotiation.cc:407] Connection header received
0925 13:09:30.806642 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:30.806643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:30.806658 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:30.806772 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:30.807368 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:30.808204 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:30.809695 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:30.809923 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:30.810008 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:30.812151 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:30.812153 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:30.812153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:30.812172 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:30.814104 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:30.814590 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:30.814591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:30.814591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:30.814652 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:30.814955 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:30.814956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:30.814956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:30.815091 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:30.816484 (+ 1393us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:30.816527 (+ 43us) server_negotiation.cc:299] Negotiation successful
0925 13:09:30.816611 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
I20260925 13:09:30.817222 13276 heartbeater.cc:347] Connected to a master server at 127.0.206.126:39597
I20260925 13:09:30.817358 13276 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
I20260925 13:09:30.817588 13276 heartbeater.cc:511] Master 127.0.206.126:39597 requested a full tablet report, sending...
I20260925 13:09:30.818231 13064 ts_manager.cc:194] Registered new tserver with Master: cf55337cdcad4caa9663ba2dd6e31b0a (127.0.206.65:40731)
I20260925 13:09:30.819201 13064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:43723
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260925 13:09:30.837461 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:54173/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39597
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:30.955729 13282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:30.955993 13282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:30.956073 13282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:30.957757 13282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:30.957818 13282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:30.957857 13282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:09:30.959971 13282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.206.84:54173/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:30.960206 13282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:30.960505 13282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Sep 25 13:09:30 dist-test-slave-chkf krb5kdc[12588](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790341770, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 13:09:30.966750 13290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:30.966753 13289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:30.966846 13292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:30.967700 13282 server_base.cc:1034] running on GCE node
I20260925 13:09:30.967952 13282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:30.968258 13282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:30.971594 13282 hybrid_clock.cc:648] HybridClock initialized: now 1790341770971568 us; error 36 us; skew 500 ppm
Sep 25 13:09:30 dist-test-slave-chkf krb5kdc[12588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341770, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:30.974876 13282 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:09:30.975485 13282 webserver.cc:460] Webserver started at http://127.0.206.66:34737/ using document root <none> and password file <none>
I20260925 13:09:30.975795 13282 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:30.975898 13282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:30.976084 13282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 13:09:30 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341770, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260925 13:09:31.003297 13282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "bdb3464d435b4d25bb500c33ce1bf4e8"
format_stamp: "Formatted at 2026-09-25 13:09:30 on dist-test-slave-chkf"
server_key: "9fa91831447d2081660fcc6382784454"
server_key_iv: "9b0039c6235815be904ddd463d8e9cfa"
server_key_version: "kuduclusterkey@0"
I20260925 13:09:31.003738 13282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "bdb3464d435b4d25bb500c33ce1bf4e8"
format_stamp: "Formatted at 2026-09-25 13:09:30 on dist-test-slave-chkf"
server_key: "9fa91831447d2081660fcc6382784454"
server_key_iv: "9b0039c6235815be904ddd463d8e9cfa"
server_key_version: "kuduclusterkey@0"
I20260925 13:09:31.005460 13282 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260925 13:09:31.018872 13299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:31.019093 13282 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260925 13:09:31.019201 13282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "bdb3464d435b4d25bb500c33ce1bf4e8"
format_stamp: "Formatted at 2026-09-25 13:09:30 on dist-test-slave-chkf"
server_key: "9fa91831447d2081660fcc6382784454"
server_key_iv: "9b0039c6235815be904ddd463d8e9cfa"
server_key_version: "kuduclusterkey@0"
I20260925 13:09:31.019318 13282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:31.042649 13282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:31.044997 13282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:31.045204 13282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:31.045482 13282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:09:31.045846 13282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:09:31.045913 13282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:31.045974 13282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:09:31.046005 13282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:31.052609 13282 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:38219
I20260925 13:09:31.053107 13282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:09:31.054697 13413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:38219 every 8 connection(s)
I20260925 13:09:31.057242 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 13282
I20260925 13:09:31.057344 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:09:31.057557 825 external_mini_cluster.cc:1428] Setting key b583321b6e570aab4c25e649a8526e7e
Sep 25 13:09:31 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341770, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:31.066461 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:31.055584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:51029 (local address 127.0.206.126:39597)
0925 13:09:31.055872 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:31.055875 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:31.056220 (+ 345us) server_negotiation.cc:407] Connection header received
0925 13:09:31.056940 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:31.056941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:31.056956 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:31.056986 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:31.058011 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:31.058827 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:31.060266 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:31.060565 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:31.060612 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:31.063272 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:31.063273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:31.063274 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:31.063281 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:31.064929 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:31.065404 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:31.065405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:31.065406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:31.065460 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:31.065721 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:31.065721 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:31.065722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:31.065849 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:31.065893 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:31.066263 (+ 370us) server_negotiation.cc:299] Negotiation successful
0925 13:09:31.066352 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255}
I20260925 13:09:31.067179 13416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:31.055581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:39597 (local address 127.0.206.66:51029)
0925 13:09:31.056065 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:31.056108 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:31.056820 (+ 712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:31.057184 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:31.057188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:31.057554 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:31.057902 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:31.057908 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:31.058937 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:31.058938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:31.060159 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:31.060162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:31.060729 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:31.060730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:31.060896 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:31.061488 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:31.061498 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:31.063161 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:31.065020 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:31.065026 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:31.065028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:31.065314 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:31.065528 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:31.065528 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:31.065529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:31.065647 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:31.065942 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:31.065947 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:31.066100 (+ 153us) client_negotiation.cc:769] Sending connection context
0925 13:09:31.066787 (+ 687us) client_negotiation.cc:240] Negotiation successful
0925 13:09:31.066943 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":77,"threads_started":1}
I20260925 13:09:31.067883 13414 heartbeater.cc:347] Connected to a master server at 127.0.206.126:39597
I20260925 13:09:31.067986 13414 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:31.068624 13414 heartbeater.cc:511] Master 127.0.206.126:39597 requested a full tablet report, sending...
I20260925 13:09:31.069010 13064 ts_manager.cc:194] Registered new tserver with Master: bdb3464d435b4d25bb500c33ce1bf4e8 (127.0.206.66:38219)
I20260925 13:09:31.069527 13064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:51029
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260925 13:09:31.102021 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:54173/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39597
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:31.224144 13419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:31.224503 13419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:31.224608 13419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:31.226353 13419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:31.226464 13419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:31.226557 13419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:09:31.228826 13419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.206.84:54173/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:31.229460 13419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:31.229830 13419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:31.236783 13425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:31.237149 13424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:31.237391 13427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:31.238463 13419 server_base.cc:1034] running on GCE node
I20260925 13:09:31.238675 13419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:31.238965 13419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:31.246438 13419 hybrid_clock.cc:648] HybridClock initialized: now 1790341771246414 us; error 38 us; skew 500 ppm
Sep 25 13:09:31 dist-test-slave-chkf krb5kdc[12588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341771, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:31.249837 13419 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:09:31.250370 13419 webserver.cc:460] Webserver started at http://127.0.206.67:43409/ using document root <none> and password file <none>
I20260925 13:09:31.250658 13419 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:31.250762 13419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:31.250900 13419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 13:09:31 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341771, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260925 13:09:31.264567 13419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "f3d233068b7542f992fbe205c78f08c9"
format_stamp: "Formatted at 2026-09-25 13:09:31 on dist-test-slave-chkf"
server_key: "1f4a5d26b2067e7d8e8bbf6021ebc9b2"
server_key_iv: "7a920a056c559a58276b1615d1afea66"
server_key_version: "kuduclusterkey@0"
I20260925 13:09:31.265152 13419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "f3d233068b7542f992fbe205c78f08c9"
format_stamp: "Formatted at 2026-09-25 13:09:31 on dist-test-slave-chkf"
server_key: "1f4a5d26b2067e7d8e8bbf6021ebc9b2"
server_key_iv: "7a920a056c559a58276b1615d1afea66"
server_key_version: "kuduclusterkey@0"
I20260925 13:09:31.268335 13419 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260925 13:09:31.281644 13434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:31.281894 13419 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:09:31.282061 13419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "f3d233068b7542f992fbe205c78f08c9"
format_stamp: "Formatted at 2026-09-25 13:09:31 on dist-test-slave-chkf"
server_key: "1f4a5d26b2067e7d8e8bbf6021ebc9b2"
server_key_iv: "7a920a056c559a58276b1615d1afea66"
server_key_version: "kuduclusterkey@0"
I20260925 13:09:31.282469 13419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:31.302999 13419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:31.305537 13419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:31.305765 13419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:31.306227 13419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:09:31.306629 13419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:09:31.306751 13419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:31.306830 13419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:09:31.306895 13419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:31.313133 13419 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:44021
I20260925 13:09:31.313613 13419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:09:31.314664 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 13419
I20260925 13:09:31.314817 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:09:31.315026 825 external_mini_cluster.cc:1428] Setting key 3560770c982c5457a4a1954a0bc1e398
I20260925 13:09:31.328538 13547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:44021 every 8 connection(s)
Sep 25 13:09:31 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341771, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:31.341933 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:31.329280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:38985 (local address 127.0.206.126:39597)
0925 13:09:31.329330 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:31.329332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:31.329570 (+ 238us) server_negotiation.cc:407] Connection header received
0925 13:09:31.330278 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:31.330279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:31.330291 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:31.330323 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:31.331199 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:31.331967 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:31.333430 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:31.333779 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:31.333827 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:31.337077 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:31.337079 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:31.337080 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:31.337087 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:31.338904 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:31.340826 (+ 1922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:31.340827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:31.340828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:31.340886 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:31.341187 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:31.341188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:31.341189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:31.341328 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:31.341383 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:31.341756 (+ 373us) server_negotiation.cc:299] Negotiation successful
0925 13:09:31.341839 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 13:09:31.342924 13550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:31.329179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:39597 (local address 127.0.206.67:38985)
0925 13:09:31.329442 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:31.329487 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:31.330153 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:31.330511 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:31.330516 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:31.330808 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:31.331086 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:31.331092 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:31.332083 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:31.332084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:31.333303 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:31.333306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:31.333930 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:31.333931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:31.334065 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:31.334945 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:31.334955 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:31.336958 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:31.339811 (+ 2853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:31.339818 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:31.339820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:31.340725 (+ 905us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:31.340958 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:31.340959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:31.340959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:31.341085 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:31.341434 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:31.341440 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:31.341598 (+ 158us) client_negotiation.cc:769] Sending connection context
0925 13:09:31.342307 (+ 709us) client_negotiation.cc:240] Negotiation successful
0925 13:09:31.342470 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":87,"threads_started":1}
I20260925 13:09:31.343565 13548 heartbeater.cc:347] Connected to a master server at 127.0.206.126:39597
I20260925 13:09:31.343657 13548 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:31.343849 13548 heartbeater.cc:511] Master 127.0.206.126:39597 requested a full tablet report, sending...
I20260925 13:09:31.344280 13064 ts_manager.cc:194] Registered new tserver with Master: f3d233068b7542f992fbe205c78f08c9 (127.0.206.67:44021)
I20260925 13:09:31.344760 13064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:38985
I20260925 13:09:31.348466 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:09:31.349220 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 13145
I20260925 13:09:31.356769 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 13282
I20260925 13:09:31.362920 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 13419
I20260925 13:09:31.368666 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 13032
I20260925 13:09:31.374445 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:39597
--webserver_interface=127.0.206.126
--webserver_port=46809
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:39597
--ranger_config_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:54173/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.206.84:54173/kms with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:31.459219 13551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:31.459448 13551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:31.459484 13551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:31.460620 13551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:09:31.460687 13551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:31.460709 13551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:31.460731 13551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:09:31.460760 13551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:09:31.462262 13551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.206.84:54173/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:39597
--ranger_config_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:39597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=46809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:31.462535 13551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:31.462816 13551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:31.467449 13556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:31.467458 13557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:31.467651 13559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:31.467877 13551 server_base.cc:1034] running on GCE node
I20260925 13:09:31.468123 13551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:31.468433 13551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:31.469614 13551 hybrid_clock.cc:648] HybridClock initialized: now 1790341771469594 us; error 33 us; skew 500 ppm
Sep 25 13:09:31 dist-test-slave-chkf krb5kdc[12588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341771, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:31.472482 13551 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:09:31.472954 13551 webserver.cc:460] Webserver started at http://127.0.206.126:46809/ using document root <none> and password file <none>
I20260925 13:09:31.473171 13551 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:31.473246 13551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 13:09:31 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341771, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260925 13:09:31.521986 13551 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:09:31.522825 13567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:31.523034 13551 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:09:31.523138 13551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "6f2abf44e39644568f29d76b85f17ee8"
format_stamp: "Formatted at 2026-09-25 13:09:29 on dist-test-slave-chkf"
server_key: "293282cf75acc529eda44a7e2959ccb4"
server_key_iv: "a5e61b77192e04dbf8d797ca5f8aa2c4"
server_key_version: "kuduclusterkey@0"
I20260925 13:09:31.523509 13551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:31.535879 13551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:31.538014 13551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:31.538166 13551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:31.543023 13551 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:39597
I20260925 13:09:31.543175 13619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:39597 every 8 connection(s)
I20260925 13:09:31.543495 13551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:09:31.544716 13620 sys_catalog.cc:263] Verifying existing consensus state
I20260925 13:09:31.545378 13620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8: Bootstrap starting.
I20260925 13:09:31.550913 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 13551
I20260925 13:09:31.551224 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:54173/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:40731
--local_ip_for_outbound_sockets=127.0.206.65
--tserver_master_addrs=127.0.206.126:39597
--webserver_port=37899
--webserver_interface=127.0.206.65
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.206.84:54173/kms with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:09:31.553875 13620 log.cc:824] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:31.555697 13620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:09:31.555984 13620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8: Bootstrap complete.
I20260925 13:09:31.558353 13620 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2abf44e39644568f29d76b85f17ee8" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39597 } }
I20260925 13:09:31.558609 13620 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f2abf44e39644568f29d76b85f17ee8, State: Initialized, Role: FOLLOWER
I20260925 13:09:31.558833 13620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2abf44e39644568f29d76b85f17ee8" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39597 } }
I20260925 13:09:31.558924 13620 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:09:31.558972 13620 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:09:31.559013 13620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:09:31.559983 13620 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2abf44e39644568f29d76b85f17ee8" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39597 } }
I20260925 13:09:31.560103 13620 leader_election.cc:302] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f2abf44e39644568f29d76b85f17ee8; no voters:
I20260925 13:09:31.560280 13620 leader_election.cc:288] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 13:09:31.560410 13624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 13:09:31.560576 13624 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [term 2 LEADER]: Becoming Leader. State: Replica: 6f2abf44e39644568f29d76b85f17ee8, State: Running, Role: LEADER
I20260925 13:09:31.560597 13620 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:09:31.561951 13624 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2abf44e39644568f29d76b85f17ee8" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39597 } }
I20260925 13:09:31.562438 13625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6f2abf44e39644568f29d76b85f17ee8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2abf44e39644568f29d76b85f17ee8" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39597 } } }
I20260925 13:09:31.562521 13625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [sys.catalog]: This master's current role is: LEADER
I20260925 13:09:31.562549 13620 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 13:09:31.562647 13625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f2abf44e39644568f29d76b85f17ee8. Latest consensus state: current_term: 2 leader_uuid: "6f2abf44e39644568f29d76b85f17ee8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2abf44e39644568f29d76b85f17ee8" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39597 } } }
I20260925 13:09:31.562691 13625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8 [sys.catalog]: This master's current role is: LEADER
I20260925 13:09:31.562953 13638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:09:31.563449 13638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:09:31.563580 13638 catalog_manager.cc:1259] Loaded cluster ID: 174d07d807aa4e9ab20a9b5835a59ff9
I20260925 13:09:31.563638 13638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:09:31.564428 13638 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:09:31.564628 13638 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8: Loaded TSK: 0
I20260925 13:09:31.564862 13638 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 13:09:31.662021 13622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:31.662294 13622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:31.662329 13622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:31.664103 13622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:31.664160 13622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:31.664201 13622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:09:31.666319 13622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.206.84:54173/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:40731
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=37899
--tserver_master_addrs=127.0.206.126:39597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:31.666536 13622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:31.666800 13622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:31.676187 13657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:31.676476 13658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:31.676213 13660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:31.677083 13622 server_base.cc:1034] running on GCE node
I20260925 13:09:31.677380 13622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:31.677671 13622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:31.678833 13622 hybrid_clock.cc:648] HybridClock initialized: now 1790341771678802 us; error 45 us; skew 500 ppm
Sep 25 13:09:31 dist-test-slave-chkf krb5kdc[12588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341771, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:31.682008 13622 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:09:31.682533 13622 webserver.cc:460] Webserver started at http://127.0.206.65:37899/ using document root <none> and password file <none>
I20260925 13:09:31.682825 13622 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:31.682910 13622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 13:09:31 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341771, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260925 13:09:31.703135 13622 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260925 13:09:31.704206 13668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:31.704380 13622 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:09:31.704491 13622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "cf55337cdcad4caa9663ba2dd6e31b0a"
format_stamp: "Formatted at 2026-09-25 13:09:30 on dist-test-slave-chkf"
server_key: "d4183f1d38f1b254ba76c826971ea228"
server_key_iv: "eb45dc1d017ee20e3d1bf3b114c42fd7"
server_key_version: "kuduclusterkey@0"
I20260925 13:09:31.704974 13622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:31.726325 13622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:31.728861 13622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:31.729001 13622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:31.729241 13622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:09:31.729549 13622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:09:31.729586 13622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:31.729615 13622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:09:31.729633 13622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:31.736393 13622 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:40731
I20260925 13:09:31.736847 13622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:09:31.739079 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 13622
I20260925 13:09:31.739383 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:54173/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:38219
--local_ip_for_outbound_sockets=127.0.206.66
--tserver_master_addrs=127.0.206.126:39597
--webserver_port=34737
--webserver_interface=127.0.206.66
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.206.84:54173/kms with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:09:31.743285 13781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:40731 every 8 connection(s)
Sep 25 13:09:31 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341771, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:31.764142 13784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:31.737623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:39597 (local address 127.0.206.65:41089)
0925 13:09:31.738155 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:31.738210 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:31.738872 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:31.747451 (+ 8579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:31.747456 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:31.747745 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:31.748077 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:31.748085 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:31.750202 (+ 2117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:31.750203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:31.751607 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:31.751610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:31.752439 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:31.752441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:31.752569 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:31.753195 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:31.753204 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:31.754782 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:31.762300 (+ 7518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:31.762308 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:31.762311 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:31.762608 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:31.762921 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:31.762922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:31.762922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:31.763039 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:31.763484 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:31.763490 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:31.763631 (+ 141us) client_negotiation.cc:769] Sending connection context
0925 13:09:31.763717 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 13:09:31.763868 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":79,"threads_started":1}
I20260925 13:09:31.764873 13785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:31.737977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:41089 (local address 127.0.206.126:39597)
0925 13:09:31.746868 (+ 8891us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:31.746880 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:31.746926 (+ 46us) server_negotiation.cc:407] Connection header received
0925 13:09:31.747099 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:31.747107 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:31.747198 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:31.748241 (+ 1043us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:31.748356 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:31.749895 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:31.751723 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:31.752324 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:31.752854 (+ 530us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:31.754901 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:31.754905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:31.754906 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:31.754918 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:31.762196 (+ 7278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:31.762696 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:31.762699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:31.762738 (+ 39us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:31.762843 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:31.763103 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:31.763104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:31.763104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:31.763399 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:31.764479 (+ 1080us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:31.764553 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 13:09:31.764666 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8825,"thread_start_us":66,"threads_started":1}
I20260925 13:09:31.765488 13782 heartbeater.cc:347] Connected to a master server at 127.0.206.126:39597
I20260925 13:09:31.765597 13782 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:31.765808 13782 heartbeater.cc:511] Master 127.0.206.126:39597 requested a full tablet report, sending...
W20260925 13:09:31.766772 13782 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:39597 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 13:09:31.766845 13782 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:31.766908 13782 heartbeater.cc:511] Master 127.0.206.126:39597 requested a full tablet report, sending...
I20260925 13:09:31.767302 13782 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:31.767392 13782 heartbeater.cc:511] Master 127.0.206.126:39597 requested a full tablet report, sending...
W20260925 13:09:31.767792 13782 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 13:09:31.848227 13786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:31.848500 13786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:31.848533 13786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:31.850237 13786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:31.850298 13786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:31.850339 13786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:09:31.852481 13786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.206.84:54173/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:38219
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=34737
--tserver_master_addrs=127.0.206.126:39597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:31.852950 13786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:31.853262 13786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:31.858983 13794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:31.858973 13792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:31.859117 13791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:31.859568 13786 server_base.cc:1034] running on GCE node
I20260925 13:09:31.859798 13786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:31.860067 13786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:31.861228 13786 hybrid_clock.cc:648] HybridClock initialized: now 1790341771861215 us; error 27 us; skew 500 ppm
Sep 25 13:09:31 dist-test-slave-chkf krb5kdc[12588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341771, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:31.864019 13786 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:09:31.864562 13786 webserver.cc:460] Webserver started at http://127.0.206.66:34737/ using document root <none> and password file <none>
I20260925 13:09:31.864825 13786 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:31.864907 13786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 13:09:31 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341771, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260925 13:09:31.886749 13786 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260925 13:09:31.888566 13802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:31.888824 13786 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260925 13:09:31.888919 13786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "bdb3464d435b4d25bb500c33ce1bf4e8"
format_stamp: "Formatted at 2026-09-25 13:09:30 on dist-test-slave-chkf"
server_key: "9fa91831447d2081660fcc6382784454"
server_key_iv: "9b0039c6235815be904ddd463d8e9cfa"
server_key_version: "kuduclusterkey@0"
I20260925 13:09:31.889320 13786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:31.918853 13786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:31.920948 13786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:31.921123 13786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:31.921375 13786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:09:31.921742 13786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:09:31.921798 13786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:31.921823 13786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:09:31.921834 13786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:31.928822 13786 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:38219
I20260925 13:09:31.928916 13915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:38219 every 8 connection(s)
I20260925 13:09:31.929247 13786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:09:31.930812 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 13786
I20260925 13:09:31.931097 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:54173/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:44021
--local_ip_for_outbound_sockets=127.0.206.67
--tserver_master_addrs=127.0.206.126:39597
--webserver_port=43409
--webserver_interface=127.0.206.67
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.206.84:54173/kms with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 13:09:31 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341771, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:31.939476 13785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:31.929984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:41881 (local address 127.0.206.126:39597)
0925 13:09:31.930050 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:31.930051 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:31.930348 (+ 297us) server_negotiation.cc:407] Connection header received
0925 13:09:31.931035 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:31.931037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:31.931054 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:31.931136 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:31.931875 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:31.932676 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:31.933734 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:31.934058 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:31.934124 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:31.936360 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:31.936361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:31.936362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:31.936369 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:31.937827 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:31.938247 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:31.938248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:31.938249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:31.938372 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:31.938696 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:31.938698 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:31.938698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:31.938853 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:31.938941 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:31.939296 (+ 355us) server_negotiation.cc:299] Negotiation successful
0925 13:09:31.939385 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 13:09:31.939627 13918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:31.930022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:39597 (local address 127.0.206.66:41881)
0925 13:09:31.930205 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:31.930245 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:31.930923 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:31.931221 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:31.931226 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:31.931479 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:31.931760 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:31.931767 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:31.932795 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:31.932796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:31.933621 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:31.933624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:31.934211 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:31.934212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:31.934361 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:31.934970 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:31.934979 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:31.936261 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:31.937928 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:31.937935 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:31.937937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:31.938156 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:31.938488 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:31.938489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:31.938489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:31.938617 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:31.938925 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:31.938932 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:31.939133 (+ 201us) client_negotiation.cc:769] Sending connection context
0925 13:09:31.939250 (+ 117us) client_negotiation.cc:240] Negotiation successful
0925 13:09:31.939393 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260925 13:09:31.940294 13916 heartbeater.cc:347] Connected to a master server at 127.0.206.126:39597
I20260925 13:09:31.940407 13916 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:31.940582 13916 heartbeater.cc:511] Master 127.0.206.126:39597 requested a full tablet report, sending...
W20260925 13:09:31.941213 13916 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:39597 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 13:09:31.941294 13916 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:31.941356 13916 heartbeater.cc:511] Master 127.0.206.126:39597 requested a full tablet report, sending...
I20260925 13:09:31.941685 13916 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:31.941759 13916 heartbeater.cc:511] Master 127.0.206.126:39597 requested a full tablet report, sending...
W20260925 13:09:31.942014 13916 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 13:09:32.052215 13919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:32.052487 13919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:32.052526 13919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:32.054191 13919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:32.054257 13919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:32.054296 13919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:09:32.056428 13919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.206.84:54173/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:44021
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=43409
--tserver_master_addrs=127.0.206.126:39597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:32.056644 13919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:32.056936 13919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:32.063923 13928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:32.064071 13925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:32.063943 13926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:32.064934 13919 server_base.cc:1034] running on GCE node
I20260925 13:09:32.065174 13919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:32.065492 13919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:32.066740 13919 hybrid_clock.cc:648] HybridClock initialized: now 1790341772066728 us; error 27 us; skew 500 ppm
Sep 25 13:09:32 dist-test-slave-chkf krb5kdc[12588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341772, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:32.069955 13919 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:09:32.070458 13919 webserver.cc:460] Webserver started at http://127.0.206.67:43409/ using document root <none> and password file <none>
I20260925 13:09:32.070698 13919 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:32.070811 13919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 13:09:32 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341772, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260925 13:09:32.087935 13919 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:09:32.089622 13936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:32.089804 13919 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 13:09:32.089895 13919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "f3d233068b7542f992fbe205c78f08c9"
format_stamp: "Formatted at 2026-09-25 13:09:31 on dist-test-slave-chkf"
server_key: "1f4a5d26b2067e7d8e8bbf6021ebc9b2"
server_key_iv: "7a920a056c559a58276b1615d1afea66"
server_key_version: "kuduclusterkey@0"
I20260925 13:09:32.090258 13919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:32.105477 13919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:32.107784 13919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:32.107925 13919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:32.108142 13919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:09:32.108448 13919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:09:32.108516 13919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:32.108565 13919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:09:32.108589 13919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:32.115568 13919 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:44021
I20260925 13:09:32.115625 14049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:44021 every 8 connection(s)
I20260925 13:09:32.115957 13919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:09:32.118820 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 13919
Sep 25 13:09:32 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341772, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:32.126895 13785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:32.116669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:52119 (local address 127.0.206.126:39597)
0925 13:09:32.116713 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:32.116715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:32.116915 (+ 200us) server_negotiation.cc:407] Connection header received
0925 13:09:32.117382 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:32.117383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:32.117396 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:32.117477 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:32.117974 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:32.118789 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:32.120230 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:32.120519 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:32.120569 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:32.123002 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:32.123003 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:32.123003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:32.123011 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:32.124691 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:32.125725 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:32.125726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:32.125727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:32.125790 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:32.126071 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:32.126072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:32.126073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:32.126207 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:32.126277 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:32.126675 (+ 398us) server_negotiation.cc:299] Negotiation successful
0925 13:09:32.126779 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:09:32.127652 14052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:32.116635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:39597 (local address 127.0.206.67:52119)
0925 13:09:32.116783 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:32.116822 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:32.117281 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:32.117527 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:32.117530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:32.117685 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:32.117874 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:32.117878 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:32.118898 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:32.118899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:32.120122 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:32.120125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:32.120690 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:32.120691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:32.120823 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:32.121364 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:32.121373 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:32.122894 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:32.124778 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:32.124784 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:32.124786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:32.125627 (+ 841us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:32.125867 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:32.125868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:32.125868 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:32.126002 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:32.126338 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:32.126344 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:32.126495 (+ 151us) client_negotiation.cc:769] Sending connection context
0925 13:09:32.127194 (+ 699us) client_negotiation.cc:240] Negotiation successful
0925 13:09:32.127355 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260925 13:09:32.128336 14050 heartbeater.cc:347] Connected to a master server at 127.0.206.126:39597
I20260925 13:09:32.128443 14050 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:32.128634 14050 heartbeater.cc:511] Master 127.0.206.126:39597 requested a full tablet report, sending...
W20260925 13:09:32.129280 14050 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:39597 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 13:09:32.129359 14050 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:32.129431 14050 heartbeater.cc:511] Master 127.0.206.126:39597 requested a full tablet report, sending...
I20260925 13:09:32.129783 14050 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:32.129859 14050 heartbeater.cc:511] Master 127.0.206.126:39597 requested a full tablet report, sending...
W20260925 13:09:32.130173 14050 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 13:09:32.134311 14054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:32.119208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57588 (local address 127.0.206.126:39597)
0925 13:09:32.119316 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:32.119318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:32.119457 (+ 139us) server_negotiation.cc:407] Connection header received
0925 13:09:32.119532 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:32.119533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:32.119545 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:32.119611 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:32.120351 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:32.120832 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:32.121765 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:32.122002 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:32.122062 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:32.124570 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:32.124572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:32.124573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:32.124580 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:32.132959 (+ 8379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:32.133421 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:32.133422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:32.133422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:32.133470 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:32.133725 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:32.133726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:32.133727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:32.133842 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:32.133980 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:32.134093 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 13:09:32.134220 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":54,"threads_started":1}
Sep 25 13:09:32 dist-test-slave-chkf krb5kdc[12588](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790341772, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:32.767971 13782 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:32.768113 13782 heartbeater.cc:511] Master 127.0.206.126:39597 requested a full tablet report, sending...
I20260925 13:09:32.768515 13584 ts_manager.cc:194] Registered new tserver with Master: cf55337cdcad4caa9663ba2dd6e31b0a (127.0.206.65:40731)
I20260925 13:09:32.769516 13584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:41089
I20260925 13:09:32.942185 13916 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:32.942309 13916 heartbeater.cc:511] Master 127.0.206.126:39597 requested a full tablet report, sending...
I20260925 13:09:32.942687 13584 ts_manager.cc:194] Registered new tserver with Master: bdb3464d435b4d25bb500c33ce1bf4e8 (127.0.206.66:38219)
I20260925 13:09:32.943214 13584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:41881
Sep 25 13:09:32 dist-test-slave-chkf krb5kdc[12588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341772, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260925 13:09:33.130323 14050 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:33.130471 14050 heartbeater.cc:511] Master 127.0.206.126:39597 requested a full tablet report, sending...
I20260925 13:09:33.130810 13584 ts_manager.cc:194] Registered new tserver with Master: f3d233068b7542f992fbe205c78f08c9 (127.0.206.67:44021)
I20260925 13:09:33.131340 13584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:52119
I20260925 13:09:33.141157 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:09:33.150311 14071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:33.142175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57602 (local address 127.0.206.126:39597)
0925 13:09:33.142385 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:33.142387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:33.142542 (+ 155us) server_negotiation.cc:407] Connection header received
0925 13:09:33.142564 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:33.142565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:33.142576 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:33.142644 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:33.143329 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.144078 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.145098 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.145412 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.145473 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:33.147011 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:33.147012 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:33.147013 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:33.147020 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:33.148851 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:33.149224 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:33.149225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:33.149226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:33.149281 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:33.149460 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:33.149461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:33.149461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:33.149581 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:33.149728 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:33.150134 (+ 406us) server_negotiation.cc:299] Negotiation successful
0925 13:09:33.150210 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":56,"threads_started":1}
I20260925 13:09:33.152630 13584 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:09:33.155077 13584 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:09:33.163740 14077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:33.157789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:40731 (local address 127.0.0.1:55806)
0925 13:09:33.158749 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:33.158786 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:33.158852 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:33.159165 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:33.159167 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:33.159169 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:33.159225 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:33.159226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.160684 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.160685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:33.161992 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:33.161995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.163455 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.163456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:33.163533 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:33.163545 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 13:09:33.163607 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 13:09:33.163665 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":932,"thread_start_us":80,"threads_started":1}
I20260925 13:09:33.168031 14080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:33.158062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55806 (local address 127.0.206.65:40731)
0925 13:09:33.158968 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:33.158972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:33.158980 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:09:33.159011 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:33.159013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:33.159027 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:33.159087 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:33.159291 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.160597 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.162085 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.163301 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.163372 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:33.163424 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:33.167917 (+ 4493us) server_negotiation.cc:299] Negotiation successful
0925 13:09:33.167956 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":876,"thread_start_us":55,"threads_started":1}
I20260925 13:09:33.168428 14077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:33.158016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:44021 (local address 127.0.0.1:43896)
0925 13:09:33.164068 (+ 6052us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:33.164077 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:33.164108 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:33.164393 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:33.164394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:33.164395 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:33.164421 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:33.164422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.165880 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.165881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:33.166956 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:33.166958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.168223 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.168224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:33.168283 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:33.168286 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:09:33.168337 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 13:09:33.168369 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6042,"thread_start_us":68,"threads_started":1}
I20260925 13:09:33.168974 14081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:33.158131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43896 (local address 127.0.206.67:44021)
0925 13:09:33.158659 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:33.158666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:33.164162 (+ 5496us) server_negotiation.cc:407] Connection header received
0925 13:09:33.164234 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:33.164236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:33.164252 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:33.164318 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:33.164486 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.165802 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.167044 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.167757 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.167830 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:33.167876 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:33.168861 (+ 985us) server_negotiation.cc:299] Negotiation successful
0925 13:09:33.168902 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":57,"threads_started":1}
I20260925 13:09:33.169734 13699 tablet_service.cc:1467] Processing CreateTablet for tablet 29c29904c5b4456cadbab532788cbb8e (DEFAULT_TABLE table=test-table [id=08528b5c18dd499d9ee6ef96d473781c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:09:33.170061 13699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29c29904c5b4456cadbab532788cbb8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:33.174494 13965 tablet_service.cc:1467] Processing CreateTablet for tablet 29c29904c5b4456cadbab532788cbb8e (DEFAULT_TABLE table=test-table [id=08528b5c18dd499d9ee6ef96d473781c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:09:33.174826 13965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29c29904c5b4456cadbab532788cbb8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:33.178129 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:33.157803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38219 (local address 127.0.0.1:40806)
0925 13:09:33.162873 (+ 5070us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:33.162883 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:33.162923 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:33.174221 (+ 11298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:33.174222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:33.174224 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:33.174257 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:33.174258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.175700 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.175701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:33.176779 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:33.176782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.177693 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.177694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:33.177754 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:33.177757 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:09:33.177786 (+ 29us) client_negotiation.cc:240] Negotiation successful
0925 13:09:33.178064 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5027,"thread_start_us":65,"threads_started":1}
I20260925 13:09:33.178496 14082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:33.158213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40806 (local address 127.0.206.66:38219)
0925 13:09:33.174064 (+ 15851us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:33.174069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:33.174078 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:09:33.174110 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:33.174112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:33.174129 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:33.174337 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:33.174420 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.175566 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.176880 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.177576 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.178293 (+ 717us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:33.178358 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:33.178384 (+ 26us) server_negotiation.cc:299] Negotiation successful
0925 13:09:33.178423 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15817,"thread_start_us":59,"threads_started":1}
I20260925 13:09:33.179020 14084 tablet_bootstrap.cc:492] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9: Bootstrap starting.
I20260925 13:09:33.179280 13850 tablet_service.cc:1467] Processing CreateTablet for tablet 29c29904c5b4456cadbab532788cbb8e (DEFAULT_TABLE table=test-table [id=08528b5c18dd499d9ee6ef96d473781c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:09:33.179595 13850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29c29904c5b4456cadbab532788cbb8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:33.179708 14084 tablet_bootstrap.cc:654] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:33.179991 14084 log.cc:824] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:33.183525 14083 tablet_bootstrap.cc:492] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a: Bootstrap starting.
I20260925 13:09:33.184487 14083 tablet_bootstrap.cc:654] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:33.184852 14083 log.cc:824] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:33.185192 14086 tablet_bootstrap.cc:492] T 29c29904c5b4456cadbab532788cbb8e P bdb3464d435b4d25bb500c33ce1bf4e8: Bootstrap starting.
I20260925 13:09:33.185799 14086 tablet_bootstrap.cc:654] T 29c29904c5b4456cadbab532788cbb8e P bdb3464d435b4d25bb500c33ce1bf4e8: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:33.186115 14086 log.cc:824] T 29c29904c5b4456cadbab532788cbb8e P bdb3464d435b4d25bb500c33ce1bf4e8: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:33.186902 14084 tablet_bootstrap.cc:492] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9: No bootstrap required, opened a new log
I20260925 13:09:33.186964 14084 ts_tablet_manager.cc:1397] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260925 13:09:33.189409 14084 raft_consensus.cc:348] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55337cdcad4caa9663ba2dd6e31b0a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40731 } } peers { permanent_uuid: "bdb3464d435b4d25bb500c33ce1bf4e8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38219 } } peers { permanent_uuid: "f3d233068b7542f992fbe205c78f08c9" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44021 } }
I20260925 13:09:33.189560 14084 raft_consensus.cc:374] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:33.189589 14084 raft_consensus.cc:729] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d233068b7542f992fbe205c78f08c9, State: Initialized, Role: FOLLOWER
I20260925 13:09:33.189709 14084 consensus_queue.cc:260] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55337cdcad4caa9663ba2dd6e31b0a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40731 } } peers { permanent_uuid: "bdb3464d435b4d25bb500c33ce1bf4e8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38219 } } peers { permanent_uuid: "f3d233068b7542f992fbe205c78f08c9" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44021 } }
I20260925 13:09:33.189940 14084 ts_tablet_manager.cc:1428] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 13:09:33.190241 14050 heartbeater.cc:503] Master 127.0.206.126:39597 was elected leader, sending a full tablet report...
I20260925 13:09:33.191592 14083 tablet_bootstrap.cc:492] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a: No bootstrap required, opened a new log
I20260925 13:09:33.191675 14083 ts_tablet_manager.cc:1397] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260925 13:09:33.194166 14083 raft_consensus.cc:348] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55337cdcad4caa9663ba2dd6e31b0a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40731 } } peers { permanent_uuid: "bdb3464d435b4d25bb500c33ce1bf4e8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38219 } } peers { permanent_uuid: "f3d233068b7542f992fbe205c78f08c9" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44021 } }
I20260925 13:09:33.194307 14083 raft_consensus.cc:374] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:33.194339 14083 raft_consensus.cc:729] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf55337cdcad4caa9663ba2dd6e31b0a, State: Initialized, Role: FOLLOWER
I20260925 13:09:33.194814 14083 consensus_queue.cc:260] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55337cdcad4caa9663ba2dd6e31b0a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40731 } } peers { permanent_uuid: "bdb3464d435b4d25bb500c33ce1bf4e8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38219 } } peers { permanent_uuid: "f3d233068b7542f992fbe205c78f08c9" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44021 } }
I20260925 13:09:33.195122 14083 ts_tablet_manager.cc:1428] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 13:09:33.195269 13782 heartbeater.cc:503] Master 127.0.206.126:39597 was elected leader, sending a full tablet report...
I20260925 13:09:33.196065 14086 tablet_bootstrap.cc:492] T 29c29904c5b4456cadbab532788cbb8e P bdb3464d435b4d25bb500c33ce1bf4e8: No bootstrap required, opened a new log
I20260925 13:09:33.196130 14086 ts_tablet_manager.cc:1397] T 29c29904c5b4456cadbab532788cbb8e P bdb3464d435b4d25bb500c33ce1bf4e8: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260925 13:09:33.198449 14086 raft_consensus.cc:348] T 29c29904c5b4456cadbab532788cbb8e P bdb3464d435b4d25bb500c33ce1bf4e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55337cdcad4caa9663ba2dd6e31b0a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40731 } } peers { permanent_uuid: "bdb3464d435b4d25bb500c33ce1bf4e8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38219 } } peers { permanent_uuid: "f3d233068b7542f992fbe205c78f08c9" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44021 } }
I20260925 13:09:33.198618 14086 raft_consensus.cc:374] T 29c29904c5b4456cadbab532788cbb8e P bdb3464d435b4d25bb500c33ce1bf4e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:33.198652 14086 raft_consensus.cc:729] T 29c29904c5b4456cadbab532788cbb8e P bdb3464d435b4d25bb500c33ce1bf4e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdb3464d435b4d25bb500c33ce1bf4e8, State: Initialized, Role: FOLLOWER
I20260925 13:09:33.198879 14086 consensus_queue.cc:260] T 29c29904c5b4456cadbab532788cbb8e P bdb3464d435b4d25bb500c33ce1bf4e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55337cdcad4caa9663ba2dd6e31b0a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40731 } } peers { permanent_uuid: "bdb3464d435b4d25bb500c33ce1bf4e8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38219 } } peers { permanent_uuid: "f3d233068b7542f992fbe205c78f08c9" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44021 } }
I20260925 13:09:33.199170 14086 ts_tablet_manager.cc:1428] T 29c29904c5b4456cadbab532788cbb8e P bdb3464d435b4d25bb500c33ce1bf4e8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 13:09:33.199306 13916 heartbeater.cc:503] Master 127.0.206.126:39597 was elected leader, sending a full tablet report...
I20260925 13:09:33.200158 14090 raft_consensus.cc:482] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:09:33.200274 14090 raft_consensus.cc:504] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55337cdcad4caa9663ba2dd6e31b0a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40731 } } peers { permanent_uuid: "bdb3464d435b4d25bb500c33ce1bf4e8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38219 } } peers { permanent_uuid: "f3d233068b7542f992fbe205c78f08c9" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44021 } }
I20260925 13:09:33.200860 14090 leader_election.cc:288] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdb3464d435b4d25bb500c33ce1bf4e8 (127.0.206.66:38219), f3d233068b7542f992fbe205c78f08c9 (127.0.206.67:44021)
I20260925 13:09:33.204952 14081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:33.201101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:59875 (local address 127.0.206.67:44021)
0925 13:09:33.201151 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:33.201153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:33.201682 (+ 529us) server_negotiation.cc:407] Connection header received
0925 13:09:33.201773 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:33.201774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:33.201783 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:33.201812 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:33.201985 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.202658 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.204033 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.204572 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.204630 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:33.204638 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:33.204852 (+ 214us) server_negotiation.cc:299] Negotiation successful
0925 13:09:33.204889 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 13:09:33.205327 14093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:33.200959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:44021 (local address 127.0.206.65:59875)
0925 13:09:33.201631 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:33.201640 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:33.201703 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:33.201870 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:33.201871 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:33.201872 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:33.201916 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:33.201917 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.202767 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.202768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:33.203931 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:33.203934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.204712 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.204713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:33.204777 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:33.204780 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:09:33.205188 (+ 408us) client_negotiation.cc:240] Negotiation successful
0925 13:09:33.205231 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":10,"thread_start_us":68,"threads_started":1}
I20260925 13:09:33.205732 13990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29c29904c5b4456cadbab532788cbb8e" candidate_uuid: "cf55337cdcad4caa9663ba2dd6e31b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d233068b7542f992fbe205c78f08c9" is_pre_election: true
I20260925 13:09:33.205888 13990 raft_consensus.cc:2454] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf55337cdcad4caa9663ba2dd6e31b0a in term 0.
I20260925 13:09:33.206205 13669 leader_election.cc:302] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55337cdcad4caa9663ba2dd6e31b0a, f3d233068b7542f992fbe205c78f08c9; no voters:
I20260925 13:09:33.206414 14092 raft_consensus.cc:2781] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:09:33.206475 14092 raft_consensus.cc:482] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:09:33.206501 14092 raft_consensus.cc:3037] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:33.207384 14092 raft_consensus.cc:504] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55337cdcad4caa9663ba2dd6e31b0a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40731 } } peers { permanent_uuid: "bdb3464d435b4d25bb500c33ce1bf4e8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38219 } } peers { permanent_uuid: "f3d233068b7542f992fbe205c78f08c9" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44021 } }
I20260925 13:09:33.207510 14092 leader_election.cc:288] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [CANDIDATE]: Term 1 election: Requested vote from peers bdb3464d435b4d25bb500c33ce1bf4e8 (127.0.206.66:38219), f3d233068b7542f992fbe205c78f08c9 (127.0.206.67:44021)
I20260925 13:09:33.207669 14082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:33.200813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:43753 (local address 127.0.206.66:38219)
0925 13:09:33.200850 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:33.200852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:33.201516 (+ 664us) server_negotiation.cc:407] Connection header received
0925 13:09:33.201588 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:33.201589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:33.201599 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:33.201652 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:33.201855 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.202601 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.206757 (+ 4156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.207290 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.207337 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:33.207344 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:33.207567 (+ 223us) server_negotiation.cc:299] Negotiation successful
0925 13:09:33.207604 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:09:33.207892 14094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:33.200742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38219 (local address 127.0.206.65:43753)
0925 13:09:33.201454 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:33.201465 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:33.201530 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:33.201698 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:33.201699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:33.201701 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:33.201775 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:33.201777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.202696 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.202697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:33.206623 (+ 3926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:33.206626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.207402 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.207403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:33.207470 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:33.207474 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:09:33.207772 (+ 298us) client_negotiation.cc:240] Negotiation successful
0925 13:09:33.207810 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"mutex_wait_us":2771,"thread_start_us":69,"threads_started":1}
I20260925 13:09:33.208132 13990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29c29904c5b4456cadbab532788cbb8e" candidate_uuid: "cf55337cdcad4caa9663ba2dd6e31b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d233068b7542f992fbe205c78f08c9"
I20260925 13:09:33.208226 13990 raft_consensus.cc:3037] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:33.208246 13869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29c29904c5b4456cadbab532788cbb8e" candidate_uuid: "cf55337cdcad4caa9663ba2dd6e31b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdb3464d435b4d25bb500c33ce1bf4e8"
I20260925 13:09:33.208242 13870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29c29904c5b4456cadbab532788cbb8e" candidate_uuid: "cf55337cdcad4caa9663ba2dd6e31b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdb3464d435b4d25bb500c33ce1bf4e8" is_pre_election: true
I20260925 13:09:33.208376 13869 raft_consensus.cc:3037] T 29c29904c5b4456cadbab532788cbb8e P bdb3464d435b4d25bb500c33ce1bf4e8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:33.209100 13990 raft_consensus.cc:2454] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55337cdcad4caa9663ba2dd6e31b0a in term 1.
I20260925 13:09:33.209218 13869 raft_consensus.cc:2454] T 29c29904c5b4456cadbab532788cbb8e P bdb3464d435b4d25bb500c33ce1bf4e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf55337cdcad4caa9663ba2dd6e31b0a in term 1.
I20260925 13:09:33.209312 13669 leader_election.cc:302] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf55337cdcad4caa9663ba2dd6e31b0a, f3d233068b7542f992fbe205c78f08c9; no voters:
I20260925 13:09:33.209441 14092 raft_consensus.cc:2781] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:09:33.209491 14092 raft_consensus.cc:686] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [term 1 LEADER]: Becoming Leader. State: Replica: cf55337cdcad4caa9663ba2dd6e31b0a, State: Running, Role: LEADER
I20260925 13:09:33.209563 14092 consensus_queue.cc:237] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55337cdcad4caa9663ba2dd6e31b0a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40731 } } peers { permanent_uuid: "bdb3464d435b4d25bb500c33ce1bf4e8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38219 } } peers { permanent_uuid: "f3d233068b7542f992fbe205c78f08c9" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44021 } }
I20260925 13:09:33.210427 13584 catalog_manager.cc:5495] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a reported cstate change: term changed from 0 to 1, leader changed from <none> to cf55337cdcad4caa9663ba2dd6e31b0a (127.0.206.65). New cstate: current_term: 1 leader_uuid: "cf55337cdcad4caa9663ba2dd6e31b0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf55337cdcad4caa9663ba2dd6e31b0a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bdb3464d435b4d25bb500c33ce1bf4e8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d233068b7542f992fbe205c78f08c9" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44021 } health_report { overall_health: UNKNOWN } } }
I20260925 13:09:33.222833 14080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:33.219655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55818 (local address 127.0.206.65:40731)
0925 13:09:33.219693 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:33.219695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:33.219703 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:09:33.219733 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:33.219734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:33.219752 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:33.219815 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:09:33.219993 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.220657 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.221730 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:33.222004 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:33.222069 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:33.222292 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:09:33.222407 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:09:33.222487 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:33.222743 (+ 256us) server_negotiation.cc:299] Negotiation successful
0925 13:09:33.222762 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 13:09:33.224804 13990 raft_consensus.cc:1261] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9 [term 1 FOLLOWER]: Refusing update from remote peer cf55337cdcad4caa9663ba2dd6e31b0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:09:33.225149 14092 consensus_queue.cc:1035] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d233068b7542f992fbe205c78f08c9" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:09:33.227213 13869 raft_consensus.cc:1261] T 29c29904c5b4456cadbab532788cbb8e P bdb3464d435b4d25bb500c33ce1bf4e8 [term 1 FOLLOWER]: Refusing update from remote peer cf55337cdcad4caa9663ba2dd6e31b0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:09:33.227496 14092 consensus_queue.cc:1035] T 29c29904c5b4456cadbab532788cbb8e P cf55337cdcad4caa9663ba2dd6e31b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdb3464d435b4d25bb500c33ce1bf4e8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:09:33.230470 14097 mvcc.cc:205] Tried to move back new op lower bound from 7333239903126298624 to 7333239903066693632. Current Snapshot: MvccSnapshot[applied={T|T < 7333239903126298624}]
I20260925 13:09:33.231472 14100 mvcc.cc:205] Tried to move back new op lower bound from 7333239903126298624 to 7333239903066693632. Current Snapshot: MvccSnapshot[applied={T|T < 7333239903126298624}]
I20260925 13:09:33.232692 14098 mvcc.cc:205] Tried to move back new op lower bound from 7333239903126298624 to 7333239903066693632. Current Snapshot: MvccSnapshot[applied={T|T < 7333239903126298624}]
I20260925 13:09:33.235044 13584 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:09:33.235157 13584 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:09:33.236025 13584 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f2abf44e39644568f29d76b85f17ee8: Sending DeleteTablet for 3 replicas of tablet 29c29904c5b4456cadbab532788cbb8e
I20260925 13:09:33.236490 13850 tablet_service.cc:1514] Processing DeleteTablet for tablet 29c29904c5b4456cadbab532788cbb8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:09:33 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:40806
I20260925 13:09:33.236652 13965 tablet_service.cc:1514] Processing DeleteTablet for tablet 29c29904c5b4456cadbab532788cbb8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:09:33 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:43896
I20260925 13:09:33.237093 13699 tablet_service.cc:1514] Processing DeleteTablet for tablet 29c29904c5b4456cadbab532788cbb8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:09:33 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:55806
I20260925 13:09:33.237566 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 13622
I20260925 13:09:33.238834 14106 tablet_replica.cc:330] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9: stopping tablet replica
I20260925 13:09:33.238931 14106 raft_consensus.cc:2229] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:09:33.239027 14106 raft_consensus.cc:2258] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:09:33.239466 14106 ts_tablet_manager.cc:1905] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:09:33.241039 14106 ts_tablet_manager.cc:1918] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:09:33.241096 14106 log.cc:1196] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/29c29904c5b4456cadbab532788cbb8e
I20260925 13:09:33.241348 14106 ts_tablet_manager.cc:1939] T 29c29904c5b4456cadbab532788cbb8e P f3d233068b7542f992fbe205c78f08c9: Deleting consensus metadata
I20260925 13:09:33.242439 13568 catalog_manager.cc:4841] TS f3d233068b7542f992fbe205c78f08c9 (127.0.206.67:44021): tablet 29c29904c5b4456cadbab532788cbb8e (table test-table [id=08528b5c18dd499d9ee6ef96d473781c]) successfully deleted
I20260925 13:09:33.245380 14108 tablet_replica.cc:330] T 29c29904c5b4456cadbab532788cbb8e P bdb3464d435b4d25bb500c33ce1bf4e8: stopping tablet replica
I20260925 13:09:33.245486 14108 raft_consensus.cc:2229] T 29c29904c5b4456cadbab532788cbb8e P bdb3464d435b4d25bb500c33ce1bf4e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:09:33.245590 14108 raft_consensus.cc:2258] T 29c29904c5b4456cadbab532788cbb8e P bdb3464d435b4d25bb500c33ce1bf4e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:09:33.246040 14108 ts_tablet_manager.cc:1905] T 29c29904c5b4456cadbab532788cbb8e P bdb3464d435b4d25bb500c33ce1bf4e8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:09:33.246661 13571 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:40731 (error 108)
I20260925 13:09:33.247334 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 13786
I20260925 13:09:33.250331 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:33.250188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:40731 (local address 127.0.0.1:55826)
0925 13:09:33.250252 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:33.250298 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:40731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 13:09:33.250516 13571 catalog_manager.cc:4584] TS cf55337cdcad4caa9663ba2dd6e31b0a (127.0.206.65:40731): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 29c29904c5b4456cadbab532788cbb8e: Network error: Client connection negotiation failed: client connection to 127.0.206.65:40731: connect: Connection refused (error 111)
I20260925 13:09:33.254479 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 13919
I20260925 13:09:33.256290 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:33.256141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38219 (local address 127.0.0.1:40814)
0925 13:09:33.256214 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:33.256252 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.66:38219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260925 13:09:33.260162 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 13551
I20260925 13:09:33.265029 825 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 13:09:33.598937 825 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 13:09:33.599014 825 mini_ranger.cc:66] Stopping Ranger...
I20260925 13:09:33.942677 825 mini_ranger.cc:68] Stopped Ranger
2026-09-25 13:09:33.942 UTC [12618] LOG: received smart shutdown request
2026-09-25 13:09:33.946 UTC [12618] LOG: background worker "logical replication launcher" (PID 12627) exited with exit code 1
2026-09-25 13:09:33.946 UTC [12622] LOG: shutting down
2026-09-25 13:09:34.113 UTC [12618] LOG: database system is shut down
2026-09-25T13:09:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (44974 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:09:34 dist-test-slave-chkf krb5kdc[14119](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:09:34 dist-test-slave-chkf krb5kdc[14119](info): set up 2 sockets
Sep 25 13:09:34 dist-test-slave-chkf krb5kdc[14119](info): commencing operation
krb5kdc: starting...
W20260925 13:09:35.246461 825 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:09:35 dist-test-slave-chkf krb5kdc[14119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:09:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:09:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:09:35.355333 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:37451
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:41385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:37451,127.0.206.125:43815,127.0.206.124:40869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:35.432754 14132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:35.432986 14132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:35.433022 14132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:35.434104 14132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:09:35.434170 14132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:35.434192 14132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:35.434221 14132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:09:35.434252 14132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:09:35.435720 14132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:37451,127.0.206.125:43815,127.0.206.124:40869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:37451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:35.435983 14132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:35.436265 14132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:35.441810 14138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:35.441895 14140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:35.441790 14137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:35.442862 14132 server_base.cc:1034] running on GCE node
I20260925 13:09:35.443102 14132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:35.443363 14132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:35.444522 14132 hybrid_clock.cc:648] HybridClock initialized: now 1790341775444496 us; error 44 us; skew 500 ppm
Sep 25 13:09:35 dist-test-slave-chkf krb5kdc[14119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:35.447389 14132 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:09:35.447858 14132 webserver.cc:460] Webserver started at http://127.0.206.126:34651/ using document root <none> and password file <none>
I20260925 13:09:35.448092 14132 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:35.448158 14132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:35.448285 14132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:09:35.449318 14132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "39e1f01bdaff45b0a9565af0745de143"
format_stamp: "Formatted at 2026-09-25 13:09:35 on dist-test-slave-chkf"
server_key: "5d9a20ab77524c20fe92fe8f807a6fe7"
server_key_iv: "22fd2afbefd8f07a5ddaa6888e3708e0"
server_key_version: "encryptionkey@0"
I20260925 13:09:35.449669 14132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "39e1f01bdaff45b0a9565af0745de143"
format_stamp: "Formatted at 2026-09-25 13:09:35 on dist-test-slave-chkf"
server_key: "5d9a20ab77524c20fe92fe8f807a6fe7"
server_key_iv: "22fd2afbefd8f07a5ddaa6888e3708e0"
server_key_version: "encryptionkey@0"
I20260925 13:09:35.450963 14132 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 13:09:35.451599 14147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:35.451786 14132 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:35.451879 14132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "39e1f01bdaff45b0a9565af0745de143"
format_stamp: "Formatted at 2026-09-25 13:09:35 on dist-test-slave-chkf"
server_key: "5d9a20ab77524c20fe92fe8f807a6fe7"
server_key_iv: "22fd2afbefd8f07a5ddaa6888e3708e0"
server_key_version: "encryptionkey@0"
I20260925 13:09:35.451953 14132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:35.463052 14132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:35.464645 14132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:35.464787 14132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:35.468750 14132 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:37451
I20260925 13:09:35.468778 14199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:37451 every 8 connection(s)
I20260925 13:09:35.469121 14132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:09:35.469918 14200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:35.471305 14200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 13:09:35 dist-test-slave-chkf krb5kdc[14119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:35.479290 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 14132
I20260925 13:09:35.479396 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:09:35.479593 825 external_mini_cluster.cc:1428] Setting key 77b00a815d78660ad4b8d4a5aa5045cd
I20260925 13:09:35.480198 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.471846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:37451 (local address 127.0.0.1:53318)
0925 13:09:35.472054 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.472094 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:35.472563 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:35.472850 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:35.472852 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:35.473097 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:35.473239 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:35.473243 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.474236 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.474237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:35.474943 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:35.474945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.475437 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.475438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:35.475499 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.475988 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:35.475995 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:35.477295 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:35.478803 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:35.478809 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:35.478811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:35.478993 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:35.479236 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:35.479237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:35.479238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:35.479323 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:35.479804 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:35.479809 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:35.479864 (+ 55us) client_negotiation.cc:769] Sending connection context
0925 13:09:35.479937 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 13:09:35.480050 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":85,"threads_started":1}
I20260925 13:09:35.480342 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.471910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53318 (local address 127.0.206.126:37451)
0925 13:09:35.472062 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:35.472070 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:35.472194 (+ 124us) server_negotiation.cc:407] Connection header received
0925 13:09:35.472719 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:35.472721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:35.472740 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:35.472845 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:35.473336 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.474133 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.475036 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.475341 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.475470 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.477402 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:35.477408 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:35.477408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:35.477414 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:35.478684 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.479086 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.479088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.479089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.479167 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.479422 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.479423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.479424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.479725 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:35.479820 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:35.479993 (+ 173us) server_negotiation.cc:299] Negotiation successful
0925 13:09:35.480156 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":115,"thread_start_us":65,"threads_started":1}
I20260925 13:09:35.481079 14200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:09:35.481398 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.481291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43815 (local address 127.0.0.1:34710)
0925 13:09:35.481340 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.481380 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 13:09:35.481544 14151 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43815: connect: Connection refused (error 111)
I20260925 13:09:35.481922 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.481846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43815 (local address 127.0.0.1:34724)
0925 13:09:35.481883 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.481907 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260925 13:09:35.482050 14200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.125:43815: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43815: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.0.206.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.125.keytab.
Entry for principal kudu/127.0.206.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.125.keytab.
I20260925 13:09:35.501683 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.125:43815
--webserver_interface=127.0.206.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.125.keytab
--principal=kudu/127.0.206.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:41385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:37451,127.0.206.125:43815,127.0.206.124:40869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:09:35.531287 14200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } attempt: 1
I20260925 13:09:35.531764 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.531607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43815 (local address 127.0.0.1:34728)
0925 13:09:35.531677 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.531726 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 13:09:35.532156 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.532058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43815 (local address 127.0.0.1:34738)
0925 13:09:35.532105 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.532134 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 13:09:35.532294 14200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.125:43815: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43815: connect: Connection refused (error 111)
W20260925 13:09:35.581727 14207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:35.581915 14207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:35.581938 14207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:35.583099 14207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:09:35.583137 14207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:35.583153 14207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:35.583164 14207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:09:35.583176 14207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:09:35.584625 14207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:37451,127.0.206.125:43815,127.0.206.124:40869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.125.keytab
--principal=kudu/127.0.206.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.125:43815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.206.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:35.584863 14207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:35.585078 14207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:35.589694 14215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:35.589686 14212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:35.589709 14213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:35.589838 14207 server_base.cc:1034] running on GCE node
I20260925 13:09:35.590214 14207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:35.590497 14207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:35.591636 14207 hybrid_clock.cc:648] HybridClock initialized: now 1790341775591597 us; error 55 us; skew 500 ppm
Sep 25 13:09:35 dist-test-slave-chkf krb5kdc[14119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:35.593956 14207 init.cc:377] Logged in from keytab as kudu/127.0.206.125@KRBTEST.COM (short username kudu)
I20260925 13:09:35.594398 14207 webserver.cc:460] Webserver started at http://127.0.206.125:42009/ using document root <none> and password file <none>
I20260925 13:09:35.594636 14207 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:35.594681 14207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:35.594877 14207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:09:35.595840 14207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-1/data/instance:
uuid: "a312f99c884547c1a1355aec4b04e60a"
format_stamp: "Formatted at 2026-09-25 13:09:35 on dist-test-slave-chkf"
server_key: "bc2e95e7286d8374a3e0de57e982c127"
server_key_iv: "ea709fd61cbfcd9170b401e380084d8d"
server_key_version: "encryptionkey@0"
I20260925 13:09:35.596201 14207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-1/wal/instance:
uuid: "a312f99c884547c1a1355aec4b04e60a"
format_stamp: "Formatted at 2026-09-25 13:09:35 on dist-test-slave-chkf"
server_key: "bc2e95e7286d8374a3e0de57e982c127"
server_key_iv: "ea709fd61cbfcd9170b401e380084d8d"
server_key_version: "encryptionkey@0"
I20260925 13:09:35.597426 14207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:09:35.598116 14222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:35.598300 14207 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:09:35.598364 14207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-1/wal
uuid: "a312f99c884547c1a1355aec4b04e60a"
format_stamp: "Formatted at 2026-09-25 13:09:35 on dist-test-slave-chkf"
server_key: "bc2e95e7286d8374a3e0de57e982c127"
server_key_iv: "ea709fd61cbfcd9170b401e380084d8d"
server_key_version: "encryptionkey@0"
I20260925 13:09:35.598452 14207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:35.600445 14200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } attempt: 2
I20260925 13:09:35.600814 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.600674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43815 (local address 127.0.0.1:34746)
0925 13:09:35.600740 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.600779 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 13:09:35.601164 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.601066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43815 (local address 127.0.0.1:34762)
0925 13:09:35.601114 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.601141 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 13:09:35.601274 14200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.125:43815: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43815: connect: Connection refused (error 111)
I20260925 13:09:35.613960 14207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:35.615674 14207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:35.615815 14207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:35.619680 14207 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.125:43815
I20260925 13:09:35.619717 14274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.125:43815 every 8 connection(s)
I20260925 13:09:35.620023 14207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-1/data/info.pb
I20260925 13:09:35.620780 14275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:35.622182 14275 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:09:35.625844 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 14207
I20260925 13:09:35.625947 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-1/wal/instance
I20260925 13:09:35.626169 825 external_mini_cluster.cc:1428] Setting key 9604bfcd0247a95e89caf47dc3a8eb0d
Sep 25 13:09:35 dist-test-slave-chkf krb5kdc[14119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:35.631337 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.622804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53332 (local address 127.0.206.126:37451)
0925 13:09:35.622866 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:35.622869 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:35.623098 (+ 229us) server_negotiation.cc:407] Connection header received
0925 13:09:35.623475 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:35.623476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:35.623487 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:35.623538 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:35.624046 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.624493 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.625427 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.625622 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.625671 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.628320 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:35.628321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:35.628322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:35.628328 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:35.629942 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.630369 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.630370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.630371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.630435 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.630666 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.630667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.630667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.630832 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:35.630898 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:35.631215 (+ 317us) server_negotiation.cc:299] Negotiation successful
0925 13:09:35.631279 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 13:09:35.631649 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.622809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:37451 (local address 127.0.0.1:53332)
0925 13:09:35.622992 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.623022 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:35.623403 (+ 381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:35.623594 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:35.623597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:35.623813 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:35.623982 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:35.623986 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.624565 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.624565 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:35.625360 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:35.625362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.625714 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.625715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:35.625874 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.626461 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:35.626470 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:35.628230 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:35.630046 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:35.630054 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:35.630056 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:35.630290 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:35.630502 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:35.630503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:35.630503 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:35.630599 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:35.630907 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:35.630914 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:35.631070 (+ 156us) client_negotiation.cc:769] Sending connection context
0925 13:09:35.631196 (+ 126us) client_negotiation.cc:240] Negotiation successful
0925 13:09:35.631364 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260925 13:09:35.632223 14275 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.206.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.124@KRBTEST.COM" created.
Sep 25 13:09:35 dist-test-slave-chkf krb5kdc[14119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
I20260925 13:09:35.641075 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.632439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43815 (local address 127.0.0.1:34776)
0925 13:09:35.632484 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.632494 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:35.632533 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:35.632807 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:35.632807 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:35.632910 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:35.632950 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:35.632951 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.634243 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.634244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:35.634758 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:35.634759 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.635199 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.635199 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:35.635259 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.635604 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:35.635608 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:35.637986 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:35.640159 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:35.640160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:35.640160 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:35.640306 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:35.640513 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:35.640513 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:35.640514 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:35.640574 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:35.640874 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:35.640874 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:35.640905 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 13:09:35.640953 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 13:09:35.641021 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260925 13:09:35.641209 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.632491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34776 (local address 127.0.206.125:43815)
0925 13:09:35.632646 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:35.632651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:35.632676 (+ 25us) server_negotiation.cc:407] Connection header received
0925 13:09:35.632700 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:35.632702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:35.632716 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:35.632797 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:35.633023 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.634153 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.634830 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.635108 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.635199 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.638070 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:35.638077 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:35.638078 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:35.638084 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:35.640079 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.640384 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.640387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.640388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.640449 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.640632 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.640633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.640633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.640807 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:35.640893 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:35.640987 (+ 94us) server_negotiation.cc:299] Negotiation successful
0925 13:09:35.641090 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":123,"thread_start_us":75,"threads_started":1}
I20260925 13:09:35.641770 14275 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:09:35.642076 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.641963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40869 (local address 127.0.0.1:38780)
0925 13:09:35.642009 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.642050 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 13:09:35.642196 14224 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40869: connect: Connection refused (error 111)
I20260925 13:09:35.642560 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.642467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40869 (local address 127.0.0.1:38790)
0925 13:09:35.642517 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.642539 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 13:09:35.642670 14275 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:40869: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40869: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.124.keytab.
Entry for principal kudu/127.0.206.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.124.keytab.
I20260925 13:09:35.652782 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.124:40869
--webserver_interface=127.0.206.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.124.keytab
--principal=kudu/127.0.206.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:41385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:37451,127.0.206.125:43815,127.0.206.124:40869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:09:35.691900 14275 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } attempt: 1
I20260925 13:09:35.692297 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.692179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40869 (local address 127.0.0.1:38806)
0925 13:09:35.692236 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.692273 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 13:09:35.692407 14200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } attempt: 3
I20260925 13:09:35.692691 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.692580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40869 (local address 127.0.0.1:38818)
0925 13:09:35.692636 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.692666 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 13:09:35.692883 14275 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:40869: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40869: connect: Connection refused (error 111)
Sep 25 13:09:35 dist-test-slave-chkf krb5kdc[14119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
I20260925 13:09:35.699290 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.692679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43815 (local address 127.0.0.1:34782)
0925 13:09:35.692737 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.692749 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:35.692797 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:35.692928 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:35.692928 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:35.693103 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:35.693178 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:35.693180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.693856 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.693857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:35.694676 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:35.694680 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.695038 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.695039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:35.695116 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.695531 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:35.695537 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:35.696800 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:35.698391 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:35.698392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:35.698393 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:35.698551 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:35.698774 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:35.698775 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:35.698775 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:35.698827 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:35.699073 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:35.699074 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:35.699110 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 13:09:35.699157 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 13:09:35.699229 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 13:09:35.699340 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.692770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34782 (local address 127.0.206.125:43815)
0925 13:09:35.692813 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:35.692815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:35.692825 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:09:35.692848 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:35.692849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:35.692860 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:35.692922 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:35.693252 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.693777 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.694774 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.694972 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.695021 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.696877 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:35.696878 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:35.696878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:35.696885 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:35.698310 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.698624 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.698625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.698626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.698682 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.698887 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.698888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.698888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.699013 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:35.699078 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:35.699188 (+ 110us) server_negotiation.cc:299] Negotiation successful
0925 13:09:35.699260 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 13:09:35.699775 14200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:09:35.700115 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.700016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40869 (local address 127.0.0.1:38822)
0925 13:09:35.700063 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.700091 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 13:09:35.700409 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.700356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40869 (local address 127.0.0.1:38824)
0925 13:09:35.700377 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.700397 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260925 13:09:35.700503 14200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:40869: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40869: connect: Connection refused (error 111)
I20260925 13:09:35.731628 14200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } attempt: 1
I20260925 13:09:35.732059 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.731906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40869 (local address 127.0.0.1:38830)
0925 13:09:35.731967 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.732017 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 13:09:35.732395 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.732316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40869 (local address 127.0.0.1:38834)
0925 13:09:35.732343 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.732372 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260925 13:09:35.732514 14200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:40869: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40869: connect: Connection refused (error 111)
W20260925 13:09:35.733134 14282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:35.733301 14282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:35.733322 14282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:35.734413 14282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:09:35.734452 14282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:35.734467 14282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:35.734478 14282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:09:35.734491 14282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:09:35.736263 14282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:37451,127.0.206.125:43815,127.0.206.124:40869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.124.keytab
--principal=kudu/127.0.206.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.124:40869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.206.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:35.736497 14282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:35.736726 14282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:35.741206 14290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:35.741243 14288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:35.741273 14282 server_base.cc:1034] running on GCE node
W20260925 13:09:35.741237 14287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:35.741638 14282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:35.741914 14282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:35.743067 14282 hybrid_clock.cc:648] HybridClock initialized: now 1790341775743064 us; error 36 us; skew 500 ppm
Sep 25 13:09:35 dist-test-slave-chkf krb5kdc[14119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:35.745325 14282 init.cc:377] Logged in from keytab as kudu/127.0.206.124@KRBTEST.COM (short username kudu)
I20260925 13:09:35.745769 14282 webserver.cc:460] Webserver started at http://127.0.206.124:46301/ using document root <none> and password file <none>
I20260925 13:09:35.745971 14282 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:35.746033 14282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:35.746181 14282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:09:35.747234 14282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-2/data/instance:
uuid: "1e160f068bc3409d8c8ecad6d9413a52"
format_stamp: "Formatted at 2026-09-25 13:09:35 on dist-test-slave-chkf"
server_key: "e314ca72a7b8d8ac25d7525d495e6356"
server_key_iv: "7539077b2d3da8623a9a6f8e9cb385cc"
server_key_version: "encryptionkey@0"
I20260925 13:09:35.747578 14282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-2/wal/instance:
uuid: "1e160f068bc3409d8c8ecad6d9413a52"
format_stamp: "Formatted at 2026-09-25 13:09:35 on dist-test-slave-chkf"
server_key: "e314ca72a7b8d8ac25d7525d495e6356"
server_key_iv: "7539077b2d3da8623a9a6f8e9cb385cc"
server_key_version: "encryptionkey@0"
I20260925 13:09:35.748801 14282 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:09:35.749475 14297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:35.749635 14282 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:09:35.749727 14282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-2/wal
uuid: "1e160f068bc3409d8c8ecad6d9413a52"
format_stamp: "Formatted at 2026-09-25 13:09:35 on dist-test-slave-chkf"
server_key: "e314ca72a7b8d8ac25d7525d495e6356"
server_key_iv: "7539077b2d3da8623a9a6f8e9cb385cc"
server_key_version: "encryptionkey@0"
I20260925 13:09:35.749799 14282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:35.761045 14275 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } attempt: 2
I20260925 13:09:35.761404 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.761292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40869 (local address 127.0.0.1:38848)
0925 13:09:35.761347 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.761380 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 13:09:35.761731 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.761624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40869 (local address 127.0.0.1:38856)
0925 13:09:35.761679 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.761709 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260925 13:09:35.761819 14275 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:40869: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40869: connect: Connection refused (error 111)
I20260925 13:09:35.776516 14282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:35.778250 14282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:35.778406 14282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:35.782179 14282 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.124:40869
I20260925 13:09:35.782230 14349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.124:40869 every 8 connection(s)
I20260925 13:09:35.782559 14282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-2/data/info.pb
I20260925 13:09:35.783412 14350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:35.784821 14350 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:09:35.786888 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 14282
I20260925 13:09:35.786995 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790341627249256-825-0/minicluster-data/master-2/wal/instance
I20260925 13:09:35.787191 825 external_mini_cluster.cc:1428] Setting key c93ee0588d92f2860ffd78776374497c
Sep 25 13:09:35 dist-test-slave-chkf krb5kdc[14119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
Sep 25 13:09:35 dist-test-slave-chkf krb5kdc[14119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260925 13:09:35.794391 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.785462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53340 (local address 127.0.206.126:37451)
0925 13:09:35.785519 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:35.785521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:35.785682 (+ 161us) server_negotiation.cc:407] Connection header received
0925 13:09:35.786115 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:35.786116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:35.786127 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:35.786203 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:35.786754 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.787473 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.788919 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.789112 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.789189 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.791537 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:35.791538 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:35.791538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:35.791547 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:35.792947 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.793396 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.793397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.793398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.793463 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.793767 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.793768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.793768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.793900 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:35.793965 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:35.794269 (+ 304us) server_negotiation.cc:299] Negotiation successful
0925 13:09:35.794330 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 13:09:35.794598 14352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.785415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:37451 (local address 127.0.0.1:53340)
0925 13:09:35.785559 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.785597 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:35.786009 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:35.786249 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:35.786252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:35.786466 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:35.786640 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:35.786644 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.787572 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.787574 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:35.788836 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:35.788838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.789197 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.789198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:35.789350 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.789784 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:35.789794 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:35.791451 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:35.793015 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:35.793019 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:35.793021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:35.793305 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:35.793528 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:35.793529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:35.793529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:35.793696 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:35.793970 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:35.793977 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:35.794129 (+ 152us) client_negotiation.cc:769] Sending connection context
0925 13:09:35.794250 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 13:09:35.794380 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260925 13:09:35.795284 14350 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:09:35.796052 14354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.787556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38858 (local address 127.0.206.124:40869)
0925 13:09:35.787719 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:35.787724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:35.787789 (+ 65us) server_negotiation.cc:407] Connection header received
0925 13:09:35.787848 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:35.787849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:35.787864 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:35.787923 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:35.788615 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.789273 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.790251 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.790520 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.790574 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.792945 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:35.792951 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:35.792951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:35.792958 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:35.794792 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.795222 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.795224 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.795224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.795272 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.795466 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.795467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.795467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.795631 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:35.795779 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:35.795855 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 13:09:35.795947 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":113,"threads_started":1}
Sep 25 13:09:35 dist-test-slave-chkf krb5kdc[14119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
Sep 25 13:09:35 dist-test-slave-chkf krb5kdc[14119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:35.802309 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.795602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34794 (local address 127.0.206.125:43815)
0925 13:09:35.795661 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:35.795663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:35.795692 (+ 29us) server_negotiation.cc:407] Connection header received
0925 13:09:35.795753 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:35.795754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:35.795764 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:35.795831 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:35.796117 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.796662 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.797662 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.797956 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.798025 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.799813 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:35.799815 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:35.799815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:35.799822 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:35.801135 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.801537 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.801538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.801538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.801596 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.801806 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.801807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.801808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.801917 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:35.801991 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:35.802156 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 13:09:35.802225 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 13:09:35.802310 14352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.795533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43815 (local address 127.0.0.1:34794)
0925 13:09:35.795657 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.795667 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:35.795702 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:35.795835 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:35.795836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:35.795980 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:35.796035 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:35.796037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.796754 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.796755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:35.797583 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:35.797586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.798027 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.798028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:35.798075 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.798467 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:35.798472 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:35.799725 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:35.801255 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:35.801256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:35.801257 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:35.801479 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:35.801679 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:35.801680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:35.801680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:35.801735 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:35.802009 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:35.802010 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:35.802047 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 13:09:35.802132 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 13:09:35.802223 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260925 13:09:35.802652 14350 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:09:35.804857 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.796971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53348 (local address 127.0.206.126:37451)
0925 13:09:35.797003 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:35.797004 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:35.797011 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:09:35.797031 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:35.797031 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:35.797039 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:35.797087 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:35.797594 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.798259 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.799183 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.799480 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.799542 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.801806 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:35.801807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:35.801807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:35.801814 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:35.803506 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.803795 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.803796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.803796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.803850 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.804040 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.804041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.804041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.804139 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:35.804638 (+ 499us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:35.804675 (+ 37us) server_negotiation.cc:299] Negotiation successful
0925 13:09:35.804771 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Sep 25 13:09:35 dist-test-slave-chkf krb5kdc[14119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260925 13:09:35.807659 14200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } attempt: 2
I20260925 13:09:35.809422 14352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.802887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40869 (local address 127.0.0.1:38872)
0925 13:09:35.802935 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.802945 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:35.802980 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:35.803105 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:35.803105 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:35.803205 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:35.803229 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:35.803230 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.803998 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.803999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:35.804648 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:35.804650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.805117 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.805117 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:35.805173 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.805549 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:35.805553 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:35.806715 (+ 1162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:35.808358 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:35.808359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:35.808359 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:35.808576 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:35.808763 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:35.808764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:35.808764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:35.808814 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:35.809108 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:35.809108 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:35.809158 (+ 50us) client_negotiation.cc:769] Sending connection context
0925 13:09:35.809232 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 13:09:35.809334 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 13:09:35.809680 14354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.802936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38872 (local address 127.0.206.124:40869)
0925 13:09:35.802971 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:35.802973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:35.802982 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:09:35.803026 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:35.803027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:35.803038 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:35.803103 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:35.803297 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.803929 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.804720 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.805012 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.805075 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.806804 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:35.806805 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:35.806805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:35.806811 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:35.808275 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.808649 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.808651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.808651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.808698 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.808879 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.808880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.808880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.809024 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:35.809106 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:35.809267 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 13:09:35.809609 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":268,"server-negotiator.queue_time_us":18}
Sep 25 13:09:35 dist-test-slave-chkf krb5kdc[14119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
I20260925 13:09:35.811044 14350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52: Bootstrap starting.
I20260925 13:09:35.811717 14350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:35.812039 14350 log.cc:824] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52: Log is configured to *not* fsync() on all Append() calls
Sep 25 13:09:35 dist-test-slave-chkf krb5kdc[14119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260925 13:09:35.812812 14350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52: No bootstrap required, opened a new log
I20260925 13:09:35.813230 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.805485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34808 (local address 127.0.206.125:43815)
0925 13:09:35.805519 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:35.805521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:35.805532 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:09:35.805592 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:35.805593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:35.805602 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:35.805674 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:35.806139 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.806609 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.807479 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.807802 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.807873 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.810296 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:35.810297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:35.810297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:35.810304 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:35.811908 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.812303 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.812304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.812305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.812359 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.812551 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.812552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.812553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.812681 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:35.812855 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:35.813067 (+ 212us) server_negotiation.cc:299] Negotiation successful
0925 13:09:35.813148 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 13:09:35.814690 14350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } }
I20260925 13:09:35.814853 14350 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:35.814913 14350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e160f068bc3409d8c8ecad6d9413a52, State: Initialized, Role: FOLLOWER
I20260925 13:09:35.815030 14350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } }
I20260925 13:09:35.815317 14350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:09:35.815447 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:09:35.815832 14357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } } }
I20260925 13:09:35.816001 14357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:09:35.816140 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.807901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40869 (local address 127.0.0.1:38880)
0925 13:09:35.808072 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.808281 (+ 209us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:35.808344 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:35.808482 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:35.808482 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:35.808614 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:35.808648 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:35.808649 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.809958 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.809959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:35.810667 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:35.810670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.811161 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.811162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:35.811224 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.811572 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:35.811577 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:35.812992 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:35.814934 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:35.814935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:35.814936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:35.815161 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:35.815411 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:35.815412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:35.815413 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:35.815475 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:35.815761 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:35.815762 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:35.815811 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 13:09:35.815896 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 13:09:35.815991 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260925 13:09:35.816499 14355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.807932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38880 (local address 127.0.206.124:40869)
0925 13:09:35.808081 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:35.808083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:35.808331 (+ 248us) server_negotiation.cc:407] Connection header received
0925 13:09:35.808400 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:35.808401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:35.808411 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:35.808470 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:35.808726 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.809857 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.810790 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.811065 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.811142 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.813069 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:35.813070 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:35.813071 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:35.813077 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:35.814849 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.815280 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.815281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.815282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.815338 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.815547 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.815547 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.815548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.815688 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:35.815746 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:35.816261 (+ 515us) server_negotiation.cc:299] Negotiation successful
0925 13:09:35.816351 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":106,"threads_started":1}
W20260925 13:09:35.817535 14371 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:09:35.817646 14371 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:09:35.818009 14200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143: Bootstrap starting.
I20260925 13:09:35.818650 14200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:35.818974 14200 log.cc:824] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:35.819770 14200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143: No bootstrap required, opened a new log
I20260925 13:09:35.819697 14282 master_runner.cc:186] Error getting master registration for 127.0.206.126:37451: OK, instance_id { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" instance_seqno: 1790341775467680 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 13:09:35.821791 14282 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 13:09:35.821728 14200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } }
I20260925 13:09:35.821842 14200 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:35.821869 14200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39e1f01bdaff45b0a9565af0745de143, State: Initialized, Role: FOLLOWER
I20260925 13:09:35.822101 14200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } }
I20260925 13:09:35.822445 14200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:09:35.827555 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.819238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53358 (local address 127.0.206.126:37451)
0925 13:09:35.819320 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:35.819322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:35.819328 (+ 6us) server_negotiation.cc:407] Connection header received
0925 13:09:35.819345 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:35.819345 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:35.819353 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:35.819413 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:35.820091 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.820743 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.821732 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.822008 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.822067 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.823638 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:35.823639 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:35.823640 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:35.823647 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:35.825673 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.826043 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.826044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.826044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.826099 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.826309 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.826310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.826311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.826421 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:35.826515 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:35.827373 (+ 858us) server_negotiation.cc:299] Negotiation successful
0925 13:09:35.827451 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 13:09:35.826190 14132 master_runner.cc:186] Error getting master registration for 127.0.206.125:43815: OK, instance_id { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" instance_seqno: 1790341775618566 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 13:09:35.829035 14132 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 13:09:35.829942 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.817693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34812 (local address 127.0.206.125:43815)
0925 13:09:35.817732 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:35.817734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:35.822110 (+ 4376us) server_negotiation.cc:407] Connection header received
0925 13:09:35.822198 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:35.822199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:35.822208 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:35.822235 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:35.822966 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.823638 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.824594 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.824884 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.824924 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.826406 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:35.826407 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:35.826407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:35.826414 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:35.828767 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.829135 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.829136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.829136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.829186 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.829381 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.829381 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.829382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.829494 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:35.829557 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:35.829788 (+ 231us) server_negotiation.cc:299] Negotiation successful
0925 13:09:35.829859 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 13:09:35.830920 14378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } } }
I20260925 13:09:35.831041 14378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 13:09:35.831478 14389 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:09:35.831530 14389 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:09:35.833679 14355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.819248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38896 (local address 127.0.206.124:40869)
0925 13:09:35.819285 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:35.819286 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:35.821951 (+ 2665us) server_negotiation.cc:407] Connection header received
0925 13:09:35.822022 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:35.822023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:35.822032 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:35.822084 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:35.827308 (+ 5224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.828044 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.829031 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.829342 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.829388 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.830924 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:35.830925 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:35.830925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:35.830933 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:35.832476 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.832859 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.832861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.832861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.832926 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.833147 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.833148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.833148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.833260 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:35.833344 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:35.833496 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 13:09:35.833588 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:09:35.852967 14275 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } attempt: 3
Sep 25 13:09:35 dist-test-slave-chkf krb5kdc[14119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260925 13:09:35.861232 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.853278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40869 (local address 127.0.0.1:38900)
0925 13:09:35.853318 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:35.853327 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:35.853358 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:35.853691 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:35.853692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:35.853847 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:35.853908 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:35.853909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.854824 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.854825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:35.855610 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:35.855612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.856047 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.856048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:35.856122 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.856571 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:35.856576 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:35.858019 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:35.860175 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:35.860176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:35.860177 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:35.860383 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:35.860589 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:35.860590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:35.860590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:35.860668 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:35.860936 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:35.860937 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:35.860979 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 13:09:35.861050 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 13:09:35.861138 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 13:09:35.861251 14355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:35.853328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38900 (local address 127.0.206.124:40869)
0925 13:09:35.853548 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:35.853550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:35.853556 (+ 6us) server_negotiation.cc:407] Connection header received
0925 13:09:35.853573 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:35.853574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:35.853582 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:35.853682 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:35.853977 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.854739 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.855687 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:35.855969 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:35.856036 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:35.858105 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:35.858106 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:35.858106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:35.858113 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:35.859776 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.860465 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.860466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.860466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.860517 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:35.860743 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:35.860744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:35.860745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:35.860866 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:35.860932 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:35.861086 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 13:09:35.861163 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260925 13:09:35.863034 14275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a: Bootstrap starting.
I20260925 13:09:35.863720 14275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:35.864003 14275 log.cc:824] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:35.864293 14378 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:09:35.864387 14378 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } }
I20260925 13:09:35.864600 14378 leader_election.cc:288] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a312f99c884547c1a1355aec4b04e60a (127.0.206.125:43815), 1e160f068bc3409d8c8ecad6d9413a52 (127.0.206.124:40869)
I20260925 13:09:35.865182 14324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "39e1f01bdaff45b0a9565af0745de143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e160f068bc3409d8c8ecad6d9413a52" is_pre_election: true
I20260925 13:09:35.865324 14324 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39e1f01bdaff45b0a9565af0745de143 in term 0.
I20260925 13:09:35.865384 14275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a: No bootstrap required, opened a new log
I20260925 13:09:35.865758 14149 leader_election.cc:302] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e160f068bc3409d8c8ecad6d9413a52, 39e1f01bdaff45b0a9565af0745de143; no voters:
I20260925 13:09:35.865887 14378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:09:35.865938 14378 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:09:35.865962 14378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:35.866811 14378 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } }
I20260925 13:09:35.866921 14378 leader_election.cc:288] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [CANDIDATE]: Term 1 election: Requested vote from peers a312f99c884547c1a1355aec4b04e60a (127.0.206.125:43815), 1e160f068bc3409d8c8ecad6d9413a52 (127.0.206.124:40869)
I20260925 13:09:35.867338 14275 raft_consensus.cc:348] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } }
I20260925 13:09:35.867506 14324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "39e1f01bdaff45b0a9565af0745de143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e160f068bc3409d8c8ecad6d9413a52"
I20260925 13:09:35.867604 14324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:35.868388 14275 raft_consensus.cc:374] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:35.868443 14275 raft_consensus.cc:729] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a312f99c884547c1a1355aec4b04e60a, State: Initialized, Role: FOLLOWER
I20260925 13:09:35.868566 14275 consensus_queue.cc:260] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } }
I20260925 13:09:35.868752 14324 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39e1f01bdaff45b0a9565af0745de143 in term 1.
I20260925 13:09:35.868850 14275 sys_catalog.cc:564] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:09:35.868884 14249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "39e1f01bdaff45b0a9565af0745de143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a312f99c884547c1a1355aec4b04e60a" is_pre_election: true
I20260925 13:09:35.869004 14249 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39e1f01bdaff45b0a9565af0745de143 in term 0.
I20260925 13:09:35.869086 14248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "39e1f01bdaff45b0a9565af0745de143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a312f99c884547c1a1355aec4b04e60a"
I20260925 13:09:35.869129 14248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:35.869845 14248 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39e1f01bdaff45b0a9565af0745de143 in term 1.
I20260925 13:09:35.870743 14151 leader_election.cc:302] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39e1f01bdaff45b0a9565af0745de143, a312f99c884547c1a1355aec4b04e60a; no voters:
I20260925 13:09:35.870913 14378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:09:35.871049 14378 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [term 1 LEADER]: Becoming Leader. State: Replica: 39e1f01bdaff45b0a9565af0745de143, State: Running, Role: LEADER
I20260925 13:09:35.871114 14378 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } }
I20260925 13:09:35.872192 14404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39e1f01bdaff45b0a9565af0745de143. Latest consensus state: current_term: 1 leader_uuid: "39e1f01bdaff45b0a9565af0745de143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } } }
I20260925 13:09:35.872318 14404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [sys.catalog]: This master's current role is: LEADER
I20260925 13:09:35.872813 14406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:09:35.873036 14207 master_runner.cc:263] No leader master found from master a312f99c884547c1a1355aec4b04e60a
I20260925 13:09:35.873046 14406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:09:35.873078 14207 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 13:09:35.874377 14324 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52 [term 1 FOLLOWER]: Refusing update from remote peer 39e1f01bdaff45b0a9565af0745de143: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:09:35.874393 14248 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a [term 1 FOLLOWER]: Refusing update from remote peer 39e1f01bdaff45b0a9565af0745de143: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:09:35.874637 14404 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:09:35.874735 14378 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:09:35.875797 14357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39e1f01bdaff45b0a9565af0745de143. Latest consensus state: current_term: 1 leader_uuid: "39e1f01bdaff45b0a9565af0745de143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } } }
I20260925 13:09:35.875890 14357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:09:35.876129 14408 sys_catalog.cc:455] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39e1f01bdaff45b0a9565af0745de143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } } }
I20260925 13:09:35.876581 14404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "39e1f01bdaff45b0a9565af0745de143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } } }
I20260925 13:09:35.876652 14404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [sys.catalog]: This master's current role is: LEADER
I20260925 13:09:35.876825 14407 mvcc.cc:205] Tried to move back new op lower bound from 7333239913979449344 to 7333239913968525312. Current Snapshot: MvccSnapshot[applied={T|T < 7333239913979449344}]
I20260925 13:09:35.876912 14410 mvcc.cc:205] Tried to move back new op lower bound from 7333239913979449344 to 7333239913968525312. Current Snapshot: MvccSnapshot[applied={T|T < 7333239913979449344}]
I20260925 13:09:35.876919 14408 sys_catalog.cc:458] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:09:35.877092 14408 sys_catalog.cc:455] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39e1f01bdaff45b0a9565af0745de143. Latest consensus state: current_term: 1 leader_uuid: "39e1f01bdaff45b0a9565af0745de143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } } }
I20260925 13:09:35.877149 14408 sys_catalog.cc:458] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:09:35.877241 14408 sys_catalog.cc:455] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "39e1f01bdaff45b0a9565af0745de143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } } }
I20260925 13:09:35.877292 14408 sys_catalog.cc:458] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:09:35.877626 14406 catalog_manager.cc:1347] Generated new cluster ID: 4b80d2117ff2461aa2257ac8366b1b39
I20260925 13:09:35.877705 14406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260925 13:09:35.879603 14403 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:09:35.879669 14403 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:09:35.880820 14378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "39e1f01bdaff45b0a9565af0745de143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } } }
I20260925 13:09:35.880903 14378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143 [sys.catalog]: This master's current role is: LEADER
I20260925 13:09:35.881237 14357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "39e1f01bdaff45b0a9565af0745de143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e1f01bdaff45b0a9565af0745de143" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37451 } } peers { permanent_uuid: "a312f99c884547c1a1355aec4b04e60a" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43815 } } peers { permanent_uuid: "1e160f068bc3409d8c8ecad6d9413a52" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40869 } } }
I20260925 13:09:35.881309 14357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:09:35.881404 14409 mvcc.cc:205] Tried to move back new op lower bound from 7333239913979449344 to 7333239913968525312. Current Snapshot: MvccSnapshot[applied={T|T < 7333239913979449344}]
I20260925 13:09:35.901276 14406 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:09:35.901965 14406 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:09:35.906247 14406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39e1f01bdaff45b0a9565af0745de143: Generated new TSK 0
I20260925 13:09:35.906443 14406 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:09:36.818295 14371 catalog_manager.cc:1259] Loaded cluster ID: 4b80d2117ff2461aa2257ac8366b1b39
I20260925 13:09:36.818384 14371 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52: loading cluster ID for follower catalog manager: success
I20260925 13:09:36.819168 14371 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52: acquiring CA information for follower catalog manager: success
I20260925 13:09:36.819458 14371 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1e160f068bc3409d8c8ecad6d9413a52: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 13:09:36.880040 14403 catalog_manager.cc:1259] Loaded cluster ID: 4b80d2117ff2461aa2257ac8366b1b39
I20260925 13:09:36.880108 14403 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a: loading cluster ID for follower catalog manager: success
I20260925 13:09:36.880827 14403 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a: acquiring CA information for follower catalog manager: success
I20260925 13:09:36.881059 14403 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a312f99c884547c1a1355aec4b04e60a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 13:09:36.962134 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 14132
I20260925 13:09:36.967561 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 14207
I20260925 13:09:36.972342 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 14282
2026-09-25T13:09:36Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2752 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:09:37 dist-test-slave-chkf krb5kdc[14427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:09:37 dist-test-slave-chkf krb5kdc[14427](info): set up 2 sockets
Sep 25 13:09:37 dist-test-slave-chkf krb5kdc[14427](info): commencing operation
krb5kdc: starting...
W20260925 13:09:38.026575 825 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:09:38 dist-test-slave-chkf krb5kdc[14427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:09:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:09:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:09:38.141103 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:46441
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:46441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:38.219028 14440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:38.219213 14440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:38.219235 14440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:38.220289 14440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:09:38.220322 14440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:38.220336 14440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:38.220350 14440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:09:38.220361 14440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:09:38.221707 14440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:46441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:46441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:38.221882 14440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:38.222038 14440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:38.226483 14446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:38.226542 14448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:38.226537 14445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:38.226739 14440 server_base.cc:1034] running on GCE node
I20260925 13:09:38.226925 14440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:38.227156 14440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:38.228292 14440 hybrid_clock.cc:648] HybridClock initialized: now 1790341778228279 us; error 28 us; skew 500 ppm
Sep 25 13:09:38 dist-test-slave-chkf krb5kdc[14427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341778, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:38.230569 14440 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:09:38.230988 14440 webserver.cc:460] Webserver started at http://127.0.206.126:46867/ using document root <none> and password file <none>
I20260925 13:09:38.231163 14440 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:38.231200 14440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:38.231283 14440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:09:38.232228 14440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "9549ca4a0aa1482eacee2994e5ddf1de"
format_stamp: "Formatted at 2026-09-25 13:09:38 on dist-test-slave-chkf"
server_key: "bbd15255a56792006c0486b4c109edc8"
server_key_iv: "a525172f10435242c5ed868f2e0fdcca"
server_key_version: "encryptionkey@0"
I20260925 13:09:38.232504 14440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "9549ca4a0aa1482eacee2994e5ddf1de"
format_stamp: "Formatted at 2026-09-25 13:09:38 on dist-test-slave-chkf"
server_key: "bbd15255a56792006c0486b4c109edc8"
server_key_iv: "a525172f10435242c5ed868f2e0fdcca"
server_key_version: "encryptionkey@0"
I20260925 13:09:38.233693 14440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:09:38.234337 14455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:38.234501 14440 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:09:38.234598 14440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "9549ca4a0aa1482eacee2994e5ddf1de"
format_stamp: "Formatted at 2026-09-25 13:09:38 on dist-test-slave-chkf"
server_key: "bbd15255a56792006c0486b4c109edc8"
server_key_iv: "a525172f10435242c5ed868f2e0fdcca"
server_key_version: "encryptionkey@0"
I20260925 13:09:38.234681 14440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:38.241482 14440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:38.243300 14440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:38.243448 14440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:38.247141 14440 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:46441
I20260925 13:09:38.247176 14507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:46441 every 8 connection(s)
I20260925 13:09:38.247483 14440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:09:38.248274 14508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:38.250342 14508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de: Bootstrap starting.
I20260925 13:09:38.250919 14508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:38.251147 14508 log.cc:824] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:38.251775 14508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de: No bootstrap required, opened a new log
I20260925 13:09:38.253333 14508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9549ca4a0aa1482eacee2994e5ddf1de" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46441 } }
I20260925 13:09:38.253461 14508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:38.253484 14508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9549ca4a0aa1482eacee2994e5ddf1de, State: Initialized, Role: FOLLOWER
I20260925 13:09:38.253607 14508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9549ca4a0aa1482eacee2994e5ddf1de" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46441 } }
I20260925 13:09:38.253691 14508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:09:38.253714 14508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:09:38.253742 14508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:38.254194 14508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9549ca4a0aa1482eacee2994e5ddf1de" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46441 } }
I20260925 13:09:38.254274 14508 leader_election.cc:302] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9549ca4a0aa1482eacee2994e5ddf1de; no voters:
I20260925 13:09:38.254390 14508 leader_election.cc:288] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:09:38.254460 14511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:09:38.254621 14511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de [term 1 LEADER]: Becoming Leader. State: Replica: 9549ca4a0aa1482eacee2994e5ddf1de, State: Running, Role: LEADER
I20260925 13:09:38.254654 14508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:09:38.254736 14511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9549ca4a0aa1482eacee2994e5ddf1de" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46441 } }
I20260925 13:09:38.255157 14513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9549ca4a0aa1482eacee2994e5ddf1de. Latest consensus state: current_term: 1 leader_uuid: "9549ca4a0aa1482eacee2994e5ddf1de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9549ca4a0aa1482eacee2994e5ddf1de" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46441 } } }
I20260925 13:09:38.255290 14513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de [sys.catalog]: This master's current role is: LEADER
I20260925 13:09:38.255442 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 14440
I20260925 13:09:38.255542 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:09:38.255707 14520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:09:38.255776 825 external_mini_cluster.cc:1428] Setting key 91fb787f8f4db82a462eac9eeb23c7e2
I20260925 13:09:38.255981 14512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9549ca4a0aa1482eacee2994e5ddf1de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9549ca4a0aa1482eacee2994e5ddf1de" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46441 } } }
I20260925 13:09:38.256131 14512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de [sys.catalog]: This master's current role is: LEADER
I20260925 13:09:38.256230 14520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:09:38.257922 14520 catalog_manager.cc:1347] Generated new cluster ID: 0977ac73a008451892794501598af57d
I20260925 13:09:38.257965 14520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 13:09:38 dist-test-slave-chkf krb5kdc[14427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:38.268419 14530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:38.256501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48814 (local address 127.0.206.126:46441)
0925 13:09:38.256759 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:38.256768 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:38.256799 (+ 31us) server_negotiation.cc:407] Connection header received
0925 13:09:38.256932 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:38.256938 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:38.257002 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:38.257162 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:38.258800 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:38.260156 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:38.261354 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:38.261962 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:38.262046 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:38.264767 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:38.264771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:38.264772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:38.264783 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:38.266631 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:38.267012 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:38.267015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:38.267015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:38.267103 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:38.267271 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:38.267272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:38.267272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:38.267541 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:38.267657 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:38.268084 (+ 427us) server_negotiation.cc:299] Negotiation successful
0925 13:09:38.268208 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":73,"threads_started":1}
I20260925 13:09:38.269096 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:09:38.275868 14520 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:09:38.276360 14520 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:09:38.283669 14520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9549ca4a0aa1482eacee2994e5ddf1de: Generated new TSK 0
I20260925 13:09:38.283838 14520 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:09:38.322484 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 14440
2026-09-25T13:09:38Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1348 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:09:38 dist-test-slave-chkf krb5kdc[14535](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:09:38 dist-test-slave-chkf krb5kdc[14535](info): set up 2 sockets
Sep 25 13:09:38 dist-test-slave-chkf krb5kdc[14535](info): commencing operation
krb5kdc: starting...
W20260925 13:09:39.346686 825 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:09:39 dist-test-slave-chkf krb5kdc[14535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:09:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:09:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:09:39.458987 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:44859
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:44859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:39.536892 14548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:39.537083 14548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:39.537104 14548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:39.538146 14548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:09:39.538179 14548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:39.538193 14548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:39.538206 14548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:09:39.538218 14548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:09:39.539716 14548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:44859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:44859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:39.539907 14548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:39.540064 14548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:39.544777 14553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:39.544785 14556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:39.544777 14554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:39.544803 14548 server_base.cc:1034] running on GCE node
I20260925 13:09:39.545296 14548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:39.545526 14548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:39.546661 14548 hybrid_clock.cc:648] HybridClock initialized: now 1790341779546639 us; error 39 us; skew 500 ppm
Sep 25 13:09:39 dist-test-slave-chkf krb5kdc[14535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341779, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:39.552486 14548 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:09:39.552960 14548 webserver.cc:460] Webserver started at http://127.0.206.126:38369/ using document root <none> and password file <none>
I20260925 13:09:39.553146 14548 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:39.553185 14548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:39.553274 14548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:09:39.554221 14548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "79b82f96623b497483133dde7004c377"
format_stamp: "Formatted at 2026-09-25 13:09:39 on dist-test-slave-chkf"
server_key: "28ea354e848d755fa01fedb1e1b512e1"
server_key_iv: "ac7c2e601abbeda19fe6f1988edb6d4a"
server_key_version: "encryptionkey@0"
I20260925 13:09:39.554512 14548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "79b82f96623b497483133dde7004c377"
format_stamp: "Formatted at 2026-09-25 13:09:39 on dist-test-slave-chkf"
server_key: "28ea354e848d755fa01fedb1e1b512e1"
server_key_iv: "ac7c2e601abbeda19fe6f1988edb6d4a"
server_key_version: "encryptionkey@0"
I20260925 13:09:39.555886 14548 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:09:39.556547 14563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:39.556749 14548 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:09:39.556835 14548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "79b82f96623b497483133dde7004c377"
format_stamp: "Formatted at 2026-09-25 13:09:39 on dist-test-slave-chkf"
server_key: "28ea354e848d755fa01fedb1e1b512e1"
server_key_iv: "ac7c2e601abbeda19fe6f1988edb6d4a"
server_key_version: "encryptionkey@0"
I20260925 13:09:39.556908 14548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:39.573944 14548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:39.575762 14548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:39.575909 14548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:39.579696 14548 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:44859
I20260925 13:09:39.579754 14615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:44859 every 8 connection(s)
I20260925 13:09:39.580080 14548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:09:39.580981 14616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:39.583122 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 14548
I20260925 13:09:39.583211 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:09:39.583325 14616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377: Bootstrap starting.
I20260925 13:09:39.583432 825 external_mini_cluster.cc:1428] Setting key 02c01f64aea75f758a35c79bcb9f38cb
I20260925 13:09:39.584052 14616 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:39.584359 14616 log.cc:824] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:39.585116 14616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377: No bootstrap required, opened a new log
I20260925 13:09:39.587335 14616 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b82f96623b497483133dde7004c377" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44859 } }
I20260925 13:09:39.587495 14616 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:39.587540 14616 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b82f96623b497483133dde7004c377, State: Initialized, Role: FOLLOWER
I20260925 13:09:39.587668 14616 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b82f96623b497483133dde7004c377" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44859 } }
I20260925 13:09:39.587754 14616 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:09:39.587778 14616 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:09:39.587850 14616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:39.588457 14616 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b82f96623b497483133dde7004c377" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44859 } }
I20260925 13:09:39.588572 14616 leader_election.cc:302] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79b82f96623b497483133dde7004c377; no voters:
I20260925 13:09:39.588745 14616 leader_election.cc:288] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:09:39.588780 14621 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:09:39.588917 14621 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 1 LEADER]: Becoming Leader. State: Replica: 79b82f96623b497483133dde7004c377, State: Running, Role: LEADER
I20260925 13:09:39.589023 14616 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:09:39.589041 14621 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b82f96623b497483133dde7004c377" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44859 } }
I20260925 13:09:39.589401 14621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79b82f96623b497483133dde7004c377. Latest consensus state: current_term: 1 leader_uuid: "79b82f96623b497483133dde7004c377" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b82f96623b497483133dde7004c377" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44859 } } }
I20260925 13:09:39.589493 14621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [sys.catalog]: This master's current role is: LEADER
I20260925 13:09:39.589411 14623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79b82f96623b497483133dde7004c377" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b82f96623b497483133dde7004c377" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44859 } } }
I20260925 13:09:39.589625 14623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [sys.catalog]: This master's current role is: LEADER
I20260925 13:09:39.589788 14626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 25 13:09:39 dist-test-slave-chkf krb5kdc[14535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:39.590832 14626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:09:39.593657 14626 catalog_manager.cc:1347] Generated new cluster ID: 74e0400851224824942d3019ca2a06ea
I20260925 13:09:39.593712 14626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:09:39.594424 14619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:39.583922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45062 (local address 127.0.206.126:44859)
0925 13:09:39.584136 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:39.584146 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:39.584180 (+ 34us) server_negotiation.cc:407] Connection header received
0925 13:09:39.584696 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:39.584705 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:39.584778 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:39.584910 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:39.585679 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:39.586525 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:39.587672 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:39.588179 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:39.588259 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:39.591138 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:39.591142 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:39.591142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:39.591153 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:39.592782 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:39.593192 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:39.593196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:39.593196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:39.593319 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:39.593531 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:39.593532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:39.593532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:39.593804 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:39.593980 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:39.594074 (+ 94us) server_negotiation.cc:299] Negotiation successful
0925 13:09:39.594211 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":110,"threads_started":1}
I20260925 13:09:39.612890 14626 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:09:39.613392 14626 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:09:39.622370 14626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377: Generated new TSK 0
I20260925 13:09:39.622539 14626 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260925 13:09:39.650002 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:44859
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:39.727871 14642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:39.728055 14642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:39.728077 14642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:39.729136 14642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:39.729171 14642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:39.729195 14642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:09:39.731139 14642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:44859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:39.731317 14642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:39.731477 14642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:39.736222 14650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:39.736232 14647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:39.736299 14648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:39.736536 14642 server_base.cc:1034] running on GCE node
I20260925 13:09:39.736725 14642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:39.736944 14642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:39.738089 14642 hybrid_clock.cc:648] HybridClock initialized: now 1790341779738079 us; error 27 us; skew 500 ppm
Sep 25 13:09:39 dist-test-slave-chkf krb5kdc[14535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341779, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:39.740505 14642 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:09:39.740947 14642 webserver.cc:460] Webserver started at http://127.0.206.65:38629/ using document root <none> and password file <none>
I20260925 13:09:39.741163 14642 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:39.741227 14642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:39.741387 14642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:09:39.742370 14642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4"
format_stamp: "Formatted at 2026-09-25 13:09:39 on dist-test-slave-chkf"
server_key: "2b13c364abf63466b3b1ffcbb4441112"
server_key_iv: "2530d81ec86a82cd80d8f181709c3dd7"
server_key_version: "encryptionkey@0"
I20260925 13:09:39.742761 14642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4"
format_stamp: "Formatted at 2026-09-25 13:09:39 on dist-test-slave-chkf"
server_key: "2b13c364abf63466b3b1ffcbb4441112"
server_key_iv: "2530d81ec86a82cd80d8f181709c3dd7"
server_key_version: "encryptionkey@0"
I20260925 13:09:39.743989 14642 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:09:39.744674 14657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:39.744843 14642 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:09:39.744938 14642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4"
format_stamp: "Formatted at 2026-09-25 13:09:39 on dist-test-slave-chkf"
server_key: "2b13c364abf63466b3b1ffcbb4441112"
server_key_iv: "2530d81ec86a82cd80d8f181709c3dd7"
server_key_version: "encryptionkey@0"
I20260925 13:09:39.745016 14642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:39.767124 14642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:39.769357 14642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:39.769513 14642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:39.769735 14642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:09:39.770023 14642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:09:39.770076 14642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:39.770113 14642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:09:39.770141 14642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:39.775879 14642 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:41397
I20260925 13:09:39.775923 14770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:41397 every 8 connection(s)
I20260925 13:09:39.776207 14642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
Sep 25 13:09:39 dist-test-slave-chkf krb5kdc[14535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341779, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:39.784404 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 14642
I20260925 13:09:39.784530 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:09:39.784726 825 external_mini_cluster.cc:1428] Setting key 0139e94e81dc1e4c999bd5e19e6e3b38
I20260925 13:09:39.784896 14619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:39.776863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:46881 (local address 127.0.206.126:44859)
0925 13:09:39.776919 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:39.776922 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:39.777129 (+ 207us) server_negotiation.cc:407] Connection header received
0925 13:09:39.777585 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:39.777586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:39.777595 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:39.777649 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:39.778493 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:39.779042 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:39.779946 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:39.780147 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:39.780195 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:39.782074 (+ 1879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:39.782076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:39.782077 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:39.782083 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:39.783489 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:39.783930 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:39.783932 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:39.783932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:39.783990 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:39.784232 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:39.784240 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:39.784241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:39.784356 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:39.784433 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:39.784726 (+ 293us) server_negotiation.cc:299] Negotiation successful
0925 13:09:39.784807 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 13:09:39.785116 14773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:39.776844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:46881)
0925 13:09:39.777007 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:39.777046 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:39.777508 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:39.777697 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:39.777700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:39.778239 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:39.778420 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:39.778424 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:39.779115 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:39.779116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:39.779875 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:39.779877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:39.780223 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:39.780224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:39.780320 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:39.780713 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:39.780721 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:39.781974 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:39.783615 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:39.783621 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:39.783623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:39.783840 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:39.784058 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:39.784058 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:39.784059 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:39.784164 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:39.784424 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:39.784430 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:39.784575 (+ 145us) client_negotiation.cc:769] Sending connection context
0925 13:09:39.784695 (+ 120us) client_negotiation.cc:240] Negotiation successful
0925 13:09:39.784848 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260925 13:09:39.785585 14771 heartbeater.cc:347] Connected to a master server at 127.0.206.126:44859
I20260925 13:09:39.785691 14771 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:39.785869 14771 heartbeater.cc:511] Master 127.0.206.126:44859 requested a full tablet report, sending...
I20260925 13:09:39.786352 14580 ts_manager.cc:194] Registered new tserver with Master: 03a1a95e4d5b4a89b0d3d80a5c9743e4 (127.0.206.65:41397)
I20260925 13:09:39.787237 14580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:46881
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260925 13:09:39.808688 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:44859
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:39.887059 14776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:39.887249 14776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:39.887274 14776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:39.888440 14776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:39.888480 14776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:39.888507 14776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:09:39.889971 14776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:44859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:39.890146 14776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:39.890400 14776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:39.895217 14784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:39.895190 14782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:39.895318 14776 server_base.cc:1034] running on GCE node
W20260925 13:09:39.895243 14781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:39.895613 14776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:39.895851 14776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:39.897018 14776 hybrid_clock.cc:648] HybridClock initialized: now 1790341779897004 us; error 26 us; skew 500 ppm
Sep 25 13:09:39 dist-test-slave-chkf krb5kdc[14535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341779, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:39.899546 14776 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:09:39.899953 14776 webserver.cc:460] Webserver started at http://127.0.206.66:45895/ using document root <none> and password file <none>
I20260925 13:09:39.900187 14776 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:39.900251 14776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:39.900400 14776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:09:39.901381 14776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc"
format_stamp: "Formatted at 2026-09-25 13:09:39 on dist-test-slave-chkf"
server_key: "c20f391f7c1dd75081623e9d8cfeae37"
server_key_iv: "742273ebcd8efa40f16e8daf1ada1ecc"
server_key_version: "encryptionkey@0"
I20260925 13:09:39.901731 14776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc"
format_stamp: "Formatted at 2026-09-25 13:09:39 on dist-test-slave-chkf"
server_key: "c20f391f7c1dd75081623e9d8cfeae37"
server_key_iv: "742273ebcd8efa40f16e8daf1ada1ecc"
server_key_version: "encryptionkey@0"
I20260925 13:09:39.903065 14776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:09:39.903792 14791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:39.903951 14776 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:39.904054 14776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc"
format_stamp: "Formatted at 2026-09-25 13:09:39 on dist-test-slave-chkf"
server_key: "c20f391f7c1dd75081623e9d8cfeae37"
server_key_iv: "742273ebcd8efa40f16e8daf1ada1ecc"
server_key_version: "encryptionkey@0"
I20260925 13:09:39.904136 14776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:39.914642 14776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:39.916842 14776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:39.916996 14776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:39.917205 14776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:09:39.917483 14776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:09:39.917536 14776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:39.917569 14776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:09:39.917598 14776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:39.923097 14776 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:34897
I20260925 13:09:39.923134 14904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:34897 every 8 connection(s)
I20260925 13:09:39.923434 14776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
Sep 25 13:09:39 dist-test-slave-chkf krb5kdc[14535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341779, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:39.932272 14619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:39.924136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:59189 (local address 127.0.206.126:44859)
0925 13:09:39.924193 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:39.924195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:39.924354 (+ 159us) server_negotiation.cc:407] Connection header received
0925 13:09:39.924820 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:39.924821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:39.924831 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:39.924890 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:39.925765 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:39.926247 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:39.927273 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:39.927456 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:39.927510 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:39.929465 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:39.929467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:39.929467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:39.929474 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:39.930901 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:39.931318 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:39.931320 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:39.931320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:39.931380 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:39.931627 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:39.931628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:39.931628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:39.931751 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:39.931827 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:39.932091 (+ 264us) server_negotiation.cc:299] Negotiation successful
0925 13:09:39.932173 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 13:09:39.932451 14907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:39.924080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:59189)
0925 13:09:39.924237 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:39.924276 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:39.924738 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:39.924962 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:39.924965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:39.925488 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:39.925693 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:39.925698 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:39.926321 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:39.926322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:39.927199 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:39.927202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:39.927539 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:39.927539 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:39.927650 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:39.928081 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:39.928088 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:39.929353 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:39.930996 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:39.931003 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:39.931006 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:39.931228 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:39.931448 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:39.931449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:39.931450 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:39.931559 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:39.931818 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:39.931825 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:39.931960 (+ 135us) client_negotiation.cc:769] Sending connection context
0925 13:09:39.932074 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 13:09:39.932226 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260925 13:09:39.933030 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 14776
I20260925 13:09:39.933139 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:09:39.933120 14905 heartbeater.cc:347] Connected to a master server at 127.0.206.126:44859
I20260925 13:09:39.933221 14905 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:39.933343 825 external_mini_cluster.cc:1428] Setting key e82513355637fd7aab4814b7a6d4841d
I20260925 13:09:39.933415 14905 heartbeater.cc:511] Master 127.0.206.126:44859 requested a full tablet report, sending...
I20260925 13:09:39.933909 14579 ts_manager.cc:194] Registered new tserver with Master: 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66:34897)
I20260925 13:09:39.934288 14579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:59189
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260925 13:09:39.956903 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:44859
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:40.035064 14910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:40.035247 14910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:40.035269 14910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:40.036394 14910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:40.036437 14910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:40.036491 14910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:09:40.037907 14910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:44859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:40.038137 14910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:40.038371 14910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:40.043223 14915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:40.043237 14916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:40.043241 14918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:40.043485 14910 server_base.cc:1034] running on GCE node
I20260925 13:09:40.043669 14910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:40.043910 14910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:40.045044 14910 hybrid_clock.cc:648] HybridClock initialized: now 1790341780045025 us; error 36 us; skew 500 ppm
Sep 25 13:09:40 dist-test-slave-chkf krb5kdc[14535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:40.047456 14910 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:09:40.047911 14910 webserver.cc:460] Webserver started at http://127.0.206.67:45473/ using document root <none> and password file <none>
I20260925 13:09:40.048101 14910 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:40.048142 14910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:40.048234 14910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:09:40.049178 14910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "034c6426204c4846a2ea10f6233bca00"
format_stamp: "Formatted at 2026-09-25 13:09:40 on dist-test-slave-chkf"
server_key: "c4efb0230a15fcb4099ca95c99a927d4"
server_key_iv: "444d4a71286b85463ade0757f4bd76a4"
server_key_version: "encryptionkey@0"
I20260925 13:09:40.049477 14910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "034c6426204c4846a2ea10f6233bca00"
format_stamp: "Formatted at 2026-09-25 13:09:40 on dist-test-slave-chkf"
server_key: "c4efb0230a15fcb4099ca95c99a927d4"
server_key_iv: "444d4a71286b85463ade0757f4bd76a4"
server_key_version: "encryptionkey@0"
I20260925 13:09:40.050655 14910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:09:40.051427 14925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:40.051597 14910 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:09:40.051687 14910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "034c6426204c4846a2ea10f6233bca00"
format_stamp: "Formatted at 2026-09-25 13:09:40 on dist-test-slave-chkf"
server_key: "c4efb0230a15fcb4099ca95c99a927d4"
server_key_iv: "444d4a71286b85463ade0757f4bd76a4"
server_key_version: "encryptionkey@0"
I20260925 13:09:40.051771 14910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:40.077158 14910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:40.079023 14910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:40.079183 14910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:40.079429 14910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:09:40.079730 14910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:09:40.079783 14910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:40.079816 14910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:09:40.079843 14910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:40.085354 14910 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:41463
I20260925 13:09:40.085397 15038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:41463 every 8 connection(s)
I20260925 13:09:40.085737 14910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:09:40.091411 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 14910
I20260925 13:09:40.091526 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:09:40.091703 825 external_mini_cluster.cc:1428] Setting key eec59a09203fd69e23b68376b3830dfe
Sep 25 13:09:40 dist-test-slave-chkf krb5kdc[14535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:40.094835 14619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.086495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:50687 (local address 127.0.206.126:44859)
0925 13:09:40.086551 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:40.086552 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:40.086763 (+ 211us) server_negotiation.cc:407] Connection header received
0925 13:09:40.087250 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:40.087251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:40.087261 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:40.087317 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:40.088177 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.088825 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.089878 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.090066 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.090158 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.092442 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:40.092443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:40.092443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:40.092451 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:40.093622 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:40.093971 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:40.093971 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:40.093972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:40.094022 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:40.094183 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:40.094184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:40.094184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:40.094281 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:40.094330 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:40.094647 (+ 317us) server_negotiation.cc:299] Negotiation successful
0925 13:09:40.094757 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 13:09:40.094966 15041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.086447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:50687)
0925 13:09:40.086604 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:40.086643 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:40.087165 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:40.087369 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:40.087373 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:40.087924 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:40.088104 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.088108 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.088901 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.088901 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.089804 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.089807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.090135 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.090135 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.090227 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.090674 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:40.090682 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:40.092242 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:40.093688 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:40.093693 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:40.093695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:40.093905 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:40.094064 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:40.094064 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:40.094064 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:40.094142 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:40.094351 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:40.094357 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:40.094518 (+ 161us) client_negotiation.cc:769] Sending connection context
0925 13:09:40.094621 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 13:09:40.094786 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260925 13:09:40.095417 15039 heartbeater.cc:347] Connected to a master server at 127.0.206.126:44859
I20260925 13:09:40.095490 15039 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:40.095669 15039 heartbeater.cc:511] Master 127.0.206.126:44859 requested a full tablet report, sending...
I20260925 13:09:40.096026 14579 ts_manager.cc:194] Registered new tserver with Master: 034c6426204c4846a2ea10f6233bca00 (127.0.206.67:41463)
I20260925 13:09:40.096407 14579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:50687
I20260925 13:09:40.103346 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:09:40.109956 14619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.104570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45064 (local address 127.0.206.126:44859)
0925 13:09:40.104636 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:40.104638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:40.104715 (+ 77us) server_negotiation.cc:407] Connection header received
0925 13:09:40.104756 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:40.104757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:40.104765 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:40.104804 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:40.105294 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.105729 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.106376 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.106540 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.106586 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.107703 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:40.107705 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:40.107705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:40.107725 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:40.109066 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:40.109363 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:40.109364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:40.109364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:40.109409 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:40.109534 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:40.109534 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:40.109534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:40.109648 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:40.109720 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:40.109821 (+ 101us) server_negotiation.cc:299] Negotiation successful
0925 13:09:40.109873 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 13:09:40.111476 14579 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:09:40.112994 14579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:09:40.119065 15047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.115090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:41397 (local address 127.0.0.1:51974)
0925 13:09:40.115239 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:40.115277 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:40.115337 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:40.115616 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:40.115619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:40.115621 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:40.115686 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.115688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.116978 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.116979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.118040 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.118042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.118859 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.118860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.118916 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.118926 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 13:09:40.118981 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 13:09:40.119016 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":53,"threads_started":1}
I20260925 13:09:40.119114 15050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.115192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51974 (local address 127.0.206.65:41397)
0925 13:09:40.115406 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:40.115428 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:40.115441 (+ 13us) server_negotiation.cc:407] Connection header received
0925 13:09:40.115486 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:40.115488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:40.115505 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:40.115604 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:40.115757 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.116868 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.118124 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.118790 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.118861 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.118918 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:40.118995 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 13:09:40.119033 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":88,"threads_started":1}
I20260925 13:09:40.119541 15048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.115090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:41463 (local address 127.0.0.1:39118)
0925 13:09:40.115436 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:40.115444 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:40.115477 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:40.115903 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:40.115903 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:40.115905 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:40.115941 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.115942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.117130 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.117131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.118366 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.118367 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.119286 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.119287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.119351 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.119355 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:09:40.119432 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 13:09:40.119474 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":40,"spinlock_wait_cycles":768,"thread_start_us":55,"threads_started":1}
I20260925 13:09:40.119562 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.115187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39118 (local address 127.0.206.67:41463)
0925 13:09:40.115764 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:40.115768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:40.115777 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:09:40.115811 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:40.115813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:40.115826 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:40.115906 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:40.116029 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.117042 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.118452 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.119214 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.119285 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.119336 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:40.119461 (+ 125us) server_negotiation.cc:299] Negotiation successful
0925 13:09:40.119509 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":88,"threads_started":1}
I20260925 13:09:40.119890 14705 tablet_service.cc:1467] Processing CreateTablet for tablet 12f326eb910f4826a65fa7155d0f5c77 (DEFAULT_TABLE table=test-table [id=f2bb2161989146ee92a23594b89ba3ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:09:40.120281 14705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12f326eb910f4826a65fa7155d0f5c77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:40.120337 14973 tablet_service.cc:1467] Processing CreateTablet for tablet 12f326eb910f4826a65fa7155d0f5c77 (DEFAULT_TABLE table=test-table [id=f2bb2161989146ee92a23594b89ba3ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:09:40.120672 14973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12f326eb910f4826a65fa7155d0f5c77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:40.122737 15053 tablet_bootstrap.cc:492] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Bootstrap starting.
I20260925 13:09:40.123270 15054 tablet_bootstrap.cc:492] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00: Bootstrap starting.
I20260925 13:09:40.123682 15053 tablet_bootstrap.cc:654] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:40.123884 15054 tablet_bootstrap.cc:654] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:40.123907 15051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.115281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:34897 (local address 127.0.0.1:52614)
0925 13:09:40.115983 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:40.115993 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:40.116043 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:40.116258 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:40.116259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:40.116261 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:40.116302 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.116303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.117183 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.117184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.118364 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.118367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.123625 (+ 5258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.123626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.123689 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.123694 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 13:09:40.123757 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 13:09:40.123809 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"mutex_wait_us":31,"thread_start_us":55,"threads_started":1}
I20260925 13:09:40.124179 15052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.115483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52614 (local address 127.0.206.66:34897)
0925 13:09:40.115669 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:40.115674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:40.116098 (+ 424us) server_negotiation.cc:407] Connection header received
0925 13:09:40.116145 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:40.116147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:40.116159 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:40.116256 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:40.116355 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.117083 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.119370 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.123515 (+ 4145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.123591 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.123648 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:40.123854 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 13:09:40.123911 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":97,"threads_started":1}
I20260925 13:09:40.124320 15053 log.cc:824] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:40.124455 15054 log.cc:824] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:40.124852 14839 tablet_service.cc:1467] Processing CreateTablet for tablet 12f326eb910f4826a65fa7155d0f5c77 (DEFAULT_TABLE table=test-table [id=f2bb2161989146ee92a23594b89ba3ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:09:40.125039 15053 tablet_bootstrap.cc:492] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: No bootstrap required, opened a new log
I20260925 13:09:40.125073 14839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12f326eb910f4826a65fa7155d0f5c77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:40.125083 15053 ts_tablet_manager.cc:1397] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:09:40.126457 15053 raft_consensus.cc:348] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } } peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } }
I20260925 13:09:40.126585 15053 raft_consensus.cc:374] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:40.126605 15053 raft_consensus.cc:729] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a1a95e4d5b4a89b0d3d80a5c9743e4, State: Initialized, Role: FOLLOWER
I20260925 13:09:40.126688 15053 consensus_queue.cc:260] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } } peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } }
I20260925 13:09:40.127014 15053 ts_tablet_manager.cc:1428] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:09:40.127135 14771 heartbeater.cc:503] Master 127.0.206.126:44859 was elected leader, sending a full tablet report...
I20260925 13:09:40.127279 15054 tablet_bootstrap.cc:492] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00: No bootstrap required, opened a new log
I20260925 13:09:40.127336 15054 ts_tablet_manager.cc:1397] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260925 13:09:40.127573 15058 tablet_bootstrap.cc:492] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: Bootstrap starting.
I20260925 13:09:40.128192 15058 tablet_bootstrap.cc:654] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:40.128680 15058 log.cc:824] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:40.128921 15054 raft_consensus.cc:348] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } } peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } }
I20260925 13:09:40.129065 15054 raft_consensus.cc:374] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:40.129112 15054 raft_consensus.cc:729] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 034c6426204c4846a2ea10f6233bca00, State: Initialized, Role: FOLLOWER
I20260925 13:09:40.129232 15054 consensus_queue.cc:260] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } } peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } }
I20260925 13:09:40.129385 15058 tablet_bootstrap.cc:492] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: No bootstrap required, opened a new log
I20260925 13:09:40.129458 15058 ts_tablet_manager.cc:1397] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:09:40.129498 15054 ts_tablet_manager.cc:1428] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:09:40.129506 15039 heartbeater.cc:503] Master 127.0.206.126:44859 was elected leader, sending a full tablet report...
I20260925 13:09:40.131166 15058 raft_consensus.cc:348] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } } peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } }
I20260925 13:09:40.131397 15058 raft_consensus.cc:374] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:40.131439 15058 raft_consensus.cc:729] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a9f8a96e5aa4c89b0a6242e211f8dfc, State: Initialized, Role: FOLLOWER
I20260925 13:09:40.131549 15058 consensus_queue.cc:260] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } } peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } }
I20260925 13:09:40.131857 15058 ts_tablet_manager.cc:1428] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 13:09:40.131877 14905 heartbeater.cc:503] Master 127.0.206.126:44859 was elected leader, sending a full tablet report...
W20260925 13:09:40.173967 14906 tablet.cc:2369] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:09:40.276940 14772 tablet.cc:2369] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:09:40.336298 15040 tablet.cc:2369] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:09:40.366446 15060 raft_consensus.cc:482] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:09:40.366568 15060 raft_consensus.cc:504] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } } peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } }
I20260925 13:09:40.366955 15060 leader_election.cc:288] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03a1a95e4d5b4a89b0d3d80a5c9743e4 (127.0.206.65:41397), 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66:34897)
I20260925 13:09:40.369920 15050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.367271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:46675 (local address 127.0.206.65:41397)
0925 13:09:40.367334 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:40.367336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:40.367348 (+ 12us) server_negotiation.cc:407] Connection header received
0925 13:09:40.367381 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:40.367382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:40.367391 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:40.367491 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:40.367704 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.368244 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.369202 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.369560 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.369616 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.369623 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:40.369805 (+ 182us) server_negotiation.cc:299] Negotiation successful
0925 13:09:40.369850 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 13:09:40.369935 15041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.367179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:41397 (local address 127.0.206.67:46675)
0925 13:09:40.367238 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:40.367251 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:40.367298 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:40.367498 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:40.367501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:40.367503 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:40.367609 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.367611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.368330 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.368331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.369108 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.369110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.369642 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.369643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.369726 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.369730 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:09:40.369782 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 13:09:40.369815 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":29}
I20260925 13:09:40.369987 15052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.367415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:48545 (local address 127.0.206.66:34897)
0925 13:09:40.367461 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:40.367463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:40.367574 (+ 111us) server_negotiation.cc:407] Connection header received
0925 13:09:40.367612 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:40.367614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:40.367623 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:40.367680 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:40.367845 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.368313 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.369240 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.369625 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.369695 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.369704 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:40.369872 (+ 168us) server_negotiation.cc:299] Negotiation successful
0925 13:09:40.369912 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 13:09:40.370121 15062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.367311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:34897 (local address 127.0.206.67:48545)
0925 13:09:40.367483 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:40.367495 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:40.367563 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:40.367703 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:40.367703 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:40.367705 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:40.367772 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.367774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.368396 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.368397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.369118 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.369119 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.369711 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.369712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.369785 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.369789 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:09:40.369848 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 13:09:40.369883 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
I20260925 13:09:40.370309 14725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12f326eb910f4826a65fa7155d0f5c77" candidate_uuid: "034c6426204c4846a2ea10f6233bca00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" is_pre_election: true
I20260925 13:09:40.370492 14725 raft_consensus.cc:2454] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 034c6426204c4846a2ea10f6233bca00 in term 0.
I20260925 13:09:40.370548 14859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12f326eb910f4826a65fa7155d0f5c77" candidate_uuid: "034c6426204c4846a2ea10f6233bca00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" is_pre_election: true
I20260925 13:09:40.370682 14859 raft_consensus.cc:2454] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 034c6426204c4846a2ea10f6233bca00 in term 0.
I20260925 13:09:40.370745 14929 leader_election.cc:302] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 034c6426204c4846a2ea10f6233bca00, 03a1a95e4d5b4a89b0d3d80a5c9743e4; no voters:
I20260925 13:09:40.370867 15060 raft_consensus.cc:2781] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:09:40.370934 15060 raft_consensus.cc:482] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:09:40.370952 15060 raft_consensus.cc:3037] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:40.371673 15060 raft_consensus.cc:504] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } } peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } }
I20260925 13:09:40.371805 15060 leader_election.cc:288] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [CANDIDATE]: Term 1 election: Requested vote from peers 03a1a95e4d5b4a89b0d3d80a5c9743e4 (127.0.206.65:41397), 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66:34897)
I20260925 13:09:40.371974 14859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12f326eb910f4826a65fa7155d0f5c77" candidate_uuid: "034c6426204c4846a2ea10f6233bca00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc"
I20260925 13:09:40.372014 14725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12f326eb910f4826a65fa7155d0f5c77" candidate_uuid: "034c6426204c4846a2ea10f6233bca00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4"
I20260925 13:09:40.372051 14859 raft_consensus.cc:3037] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:40.372110 14725 raft_consensus.cc:3037] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:40.372779 14859 raft_consensus.cc:2454] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 034c6426204c4846a2ea10f6233bca00 in term 1.
I20260925 13:09:40.372896 14725 raft_consensus.cc:2454] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 034c6426204c4846a2ea10f6233bca00 in term 1.
I20260925 13:09:40.372936 14929 leader_election.cc:302] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 034c6426204c4846a2ea10f6233bca00, 6a9f8a96e5aa4c89b0a6242e211f8dfc; no voters:
I20260925 13:09:40.373028 15060 raft_consensus.cc:2781] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:09:40.373164 15060 raft_consensus.cc:686] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [term 1 LEADER]: Becoming Leader. State: Replica: 034c6426204c4846a2ea10f6233bca00, State: Running, Role: LEADER
I20260925 13:09:40.373265 15060 consensus_queue.cc:237] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } } peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } }
I20260925 13:09:40.374064 14580 catalog_manager.cc:5495] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 034c6426204c4846a2ea10f6233bca00 (127.0.206.67). New cstate: current_term: 1 leader_uuid: "034c6426204c4846a2ea10f6233bca00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } health_report { overall_health: UNKNOWN } } }
I20260925 13:09:40.401273 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.398931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52262 (local address 127.0.206.67:41463)
0925 13:09:40.398996 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:40.398998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:40.399009 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:09:40.399057 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:40.399060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:40.399081 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:40.399154 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:09:40.399288 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.399752 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.400490 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.400684 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.400760 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.400930 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:09:40.401022 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:09:40.401116 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:40.401135 (+ 19us) server_negotiation.cc:299] Negotiation successful
0925 13:09:40.401154 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 13:09:40.402967 14725 raft_consensus.cc:1261] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Refusing update from remote peer 034c6426204c4846a2ea10f6233bca00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:09:40.403055 14859 raft_consensus.cc:1261] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 1 FOLLOWER]: Refusing update from remote peer 034c6426204c4846a2ea10f6233bca00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:09:40.403291 15063 consensus_queue.cc:1035] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:09:40.403472 15060 consensus_queue.cc:1035] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:09:40.406179 15068 mvcc.cc:205] Tried to move back new op lower bound from 7333239932527894528 to 7333239932409360384. Current Snapshot: MvccSnapshot[applied={T|T < 7333239932527894528}]
I20260925 13:09:40.408183 14580 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:09:40.408280 14580 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:09:40.409036 14580 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377: Sending DeleteTablet for 3 replicas of tablet 12f326eb910f4826a65fa7155d0f5c77
I20260925 13:09:40.409355 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 14642
I20260925 13:09:40.409443 14839 tablet_service.cc:1514] Processing DeleteTablet for tablet 12f326eb910f4826a65fa7155d0f5c77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:09:40 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:52614
I20260925 13:09:40.409448 14705 tablet_service.cc:1514] Processing DeleteTablet for tablet 12f326eb910f4826a65fa7155d0f5c77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:09:40 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:51974
I20260925 13:09:40.410017 15076 tablet_replica.cc:330] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: stopping tablet replica
I20260925 13:09:40.410048 14973 tablet_service.cc:1514] Processing DeleteTablet for tablet 12f326eb910f4826a65fa7155d0f5c77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:09:40 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:39118
I20260925 13:09:40.410108 15076 raft_consensus.cc:2229] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:09:40.410223 15076 raft_consensus.cc:2258] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:09:40.410246 15077 tablet_replica.cc:330] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00: stopping tablet replica
I20260925 13:09:40.410319 15077 raft_consensus.cc:2229] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:09:40.410398 15077 raft_consensus.cc:2258] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:09:40.410661 15076 ts_tablet_manager.cc:1905] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:09:40.412444 15077 ts_tablet_manager.cc:1905] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:09:40.412592 15076 ts_tablet_manager.cc:1918] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:09:40.412671 15076 log.cc:1196] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/12f326eb910f4826a65fa7155d0f5c77
I20260925 13:09:40.412918 15076 ts_tablet_manager.cc:1939] T 12f326eb910f4826a65fa7155d0f5c77 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: Deleting consensus metadata
I20260925 13:09:40.413303 14567 catalog_manager.cc:4841] TS 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66:34897): tablet 12f326eb910f4826a65fa7155d0f5c77 (table test-table [id=f2bb2161989146ee92a23594b89ba3ac]) successfully deleted
I20260925 13:09:40.414381 15077 ts_tablet_manager.cc:1918] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:09:40.414458 15077 log.cc:1196] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/12f326eb910f4826a65fa7155d0f5c77
I20260925 13:09:40.414731 15077 ts_tablet_manager.cc:1939] T 12f326eb910f4826a65fa7155d0f5c77 P 034c6426204c4846a2ea10f6233bca00: Deleting consensus metadata
I20260925 13:09:40.415134 14565 catalog_manager.cc:4841] TS 034c6426204c4846a2ea10f6233bca00 (127.0.206.67:41463): tablet 12f326eb910f4826a65fa7155d0f5c77 (table test-table [id=f2bb2161989146ee92a23594b89ba3ac]) successfully deleted
W20260925 13:09:40.415891 14567 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:41397 (error 108)
I20260925 13:09:40.416137 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 14776
I20260925 13:09:40.416457 15051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.416343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:41397 (local address 127.0.0.1:37952)
0925 13:09:40.416385 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:40.416423 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:41397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260925 13:09:40.416569 14567 catalog_manager.cc:4584] TS 03a1a95e4d5b4a89b0d3d80a5c9743e4 (127.0.206.65:41397): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 12f326eb910f4826a65fa7155d0f5c77: Network error: Client connection negotiation failed: client connection to 127.0.206.65:41397: connect: Connection refused (error 111)
I20260925 13:09:40.421415 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 14910
I20260925 13:09:40.425761 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 14548
I20260925 13:09:40.430958 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:44859
--webserver_interface=127.0.206.126
--webserver_port=38369
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:44859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:40.509106 15079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:40.509321 15079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:40.509357 15079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:40.510464 15079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:09:40.510529 15079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:40.510553 15079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:40.510581 15079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:09:40.510638 15079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:09:40.512055 15079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:44859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:44859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=38369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:40.512315 15079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:40.512523 15079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:40.520949 15088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:40.520964 15085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:40.520964 15086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:40.521167 15079 server_base.cc:1034] running on GCE node
I20260925 13:09:40.521406 15079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:40.521699 15079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:40.522861 15079 hybrid_clock.cc:648] HybridClock initialized: now 1790341780522848 us; error 40 us; skew 500 ppm
Sep 25 13:09:40 dist-test-slave-chkf krb5kdc[14535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:40.525228 15079 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:09:40.525674 15079 webserver.cc:460] Webserver started at http://127.0.206.126:38369/ using document root <none> and password file <none>
I20260925 13:09:40.525906 15079 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:40.525987 15079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:40.527339 15079 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:09:40.527936 15095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:40.528095 15079 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:09:40.528198 15079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "79b82f96623b497483133dde7004c377"
format_stamp: "Formatted at 2026-09-25 13:09:39 on dist-test-slave-chkf"
server_key: "28ea354e848d755fa01fedb1e1b512e1"
server_key_iv: "ac7c2e601abbeda19fe6f1988edb6d4a"
server_key_version: "encryptionkey@0"
I20260925 13:09:40.528548 15079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:40.547621 15079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:40.549804 15079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:40.549958 15079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:40.553928 15079 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:44859
I20260925 13:09:40.553958 15147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:44859 every 8 connection(s)
I20260925 13:09:40.554315 15079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:09:40.555392 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 15079
I20260925 13:09:40.555682 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:41397
--local_ip_for_outbound_sockets=127.0.206.65
--tserver_master_addrs=127.0.206.126:44859
--webserver_port=38629
--webserver_interface=127.0.206.65
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:09:40.555888 15148 sys_catalog.cc:263] Verifying existing consensus state
I20260925 13:09:40.556449 15148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377: Bootstrap starting.
I20260925 13:09:40.564534 15148 log.cc:824] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:40.566936 15148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:09:40.567243 15148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377: Bootstrap complete.
I20260925 13:09:40.569916 15148 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b82f96623b497483133dde7004c377" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44859 } }
I20260925 13:09:40.570147 15148 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b82f96623b497483133dde7004c377, State: Initialized, Role: FOLLOWER
I20260925 13:09:40.570322 15148 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b82f96623b497483133dde7004c377" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44859 } }
I20260925 13:09:40.570399 15148 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:09:40.570456 15148 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:09:40.570504 15148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:09:40.571525 15148 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b82f96623b497483133dde7004c377" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44859 } }
I20260925 13:09:40.571641 15148 leader_election.cc:302] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79b82f96623b497483133dde7004c377; no voters:
I20260925 13:09:40.571805 15148 leader_election.cc:288] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 13:09:40.571847 15152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 13:09:40.571971 15152 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 2 LEADER]: Becoming Leader. State: Replica: 79b82f96623b497483133dde7004c377, State: Running, Role: LEADER
I20260925 13:09:40.572058 15148 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:09:40.572052 15152 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b82f96623b497483133dde7004c377" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44859 } }
I20260925 13:09:40.572547 15153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "79b82f96623b497483133dde7004c377" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b82f96623b497483133dde7004c377" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44859 } } }
I20260925 13:09:40.572623 15153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [sys.catalog]: This master's current role is: LEADER
I20260925 13:09:40.572824 15158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:09:40.573388 15154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79b82f96623b497483133dde7004c377. Latest consensus state: current_term: 2 leader_uuid: "79b82f96623b497483133dde7004c377" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b82f96623b497483133dde7004c377" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44859 } } }
I20260925 13:09:40.573504 15154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [sys.catalog]: This master's current role is: LEADER
I20260925 13:09:40.573640 15158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:09:40.573748 15158 catalog_manager.cc:1259] Loaded cluster ID: 74e0400851224824942d3019ca2a06ea
I20260925 13:09:40.573796 15158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:09:40.574132 15158 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260925 13:09:40.578831 15158 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260925 13:09:40.581123 15158 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:09:40.581332 15158 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377: Loaded TSK: 0
I20260925 13:09:40.581599 15158 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 13:09:40.637915 15150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:40.638125 15150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:40.638163 15150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:40.639349 15150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:40.639410 15150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:40.639454 15150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:09:40.640988 15150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:41397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=38629
--tserver_master_addrs=127.0.206.126:44859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:40.641263 15150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:40.641494 15150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:40.646224 15178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:40.646261 15175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:40.646306 15176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:40.646643 15150 server_base.cc:1034] running on GCE node
I20260925 13:09:40.646878 15150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:40.647136 15150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:40.648324 15150 hybrid_clock.cc:648] HybridClock initialized: now 1790341780648300 us; error 34 us; skew 500 ppm
Sep 25 13:09:40 dist-test-slave-chkf krb5kdc[14535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:40.651269 15150 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:09:40.651769 15150 webserver.cc:460] Webserver started at http://127.0.206.65:38629/ using document root <none> and password file <none>
I20260925 13:09:40.652025 15150 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:40.652107 15150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:40.653741 15150 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:09:40.654467 15185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:40.654664 15150 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:09:40.654774 15150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4"
format_stamp: "Formatted at 2026-09-25 13:09:39 on dist-test-slave-chkf"
server_key: "2b13c364abf63466b3b1ffcbb4441112"
server_key_iv: "2530d81ec86a82cd80d8f181709c3dd7"
server_key_version: "encryptionkey@0"
I20260925 13:09:40.655161 15150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:40.668699 15150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:40.670969 15150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:40.671111 15150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:40.671325 15150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:09:40.671756 15192 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 13:09:40.673177 15150 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 13:09:40.673240 15150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 13:09:40.673272 15150 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 13:09:40.674101 15150 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 13:09:40.674147 15150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 13:09:40.674207 15192 tablet_bootstrap.cc:492] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Bootstrap starting.
I20260925 13:09:40.681077 15150 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:41397
I20260925 13:09:40.681257 15299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:41397 every 8 connection(s)
I20260925 13:09:40.681491 15150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:09:40.684551 15192 log.cc:824] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:40.686672 15192 tablet_bootstrap.cc:492] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:09:40.686975 15192 tablet_bootstrap.cc:492] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Bootstrap complete.
I20260925 13:09:40.688158 15192 ts_tablet_manager.cc:1397] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.010s
I20260925 13:09:40.689260 15192 raft_consensus.cc:348] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } } peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } }
Sep 25 13:09:40 dist-test-slave-chkf krb5kdc[14535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:40.689496 15192 raft_consensus.cc:729] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a1a95e4d5b4a89b0d3d80a5c9743e4, State: Initialized, Role: FOLLOWER
I20260925 13:09:40.690318 15192 consensus_queue.cc:260] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } } peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } }
I20260925 13:09:40.690601 15192 ts_tablet_manager.cc:1428] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 13:09:40.690665 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 15150
I20260925 13:09:40.690982 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:34897
--local_ip_for_outbound_sockets=127.0.206.66
--tserver_master_addrs=127.0.206.126:44859
--webserver_port=45895
--webserver_interface=127.0.206.66
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:09:40.694429 15302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.682265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:43545)
0925 13:09:40.682478 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:40.682522 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:40.683107 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:40.683616 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:40.683620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:40.683950 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:40.684199 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.684218 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.685909 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.685910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.687196 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.687199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.688095 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.688096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.688251 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.688862 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:40.688871 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:40.690548 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:40.692860 (+ 2312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:40.692867 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:40.692869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:40.693182 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:40.693501 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:40.693501 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:40.693502 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:40.693593 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:40.694024 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:40.694028 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:40.694125 (+ 97us) client_negotiation.cc:769] Sending connection context
0925 13:09:40.694199 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 13:09:40.694308 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":115,"threads_started":1}
I20260925 13:09:40.694624 15303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.682567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:43545 (local address 127.0.206.126:44859)
0925 13:09:40.682818 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:40.682827 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:40.682869 (+ 42us) server_negotiation.cc:407] Connection header received
0925 13:09:40.683347 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:40.683355 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:40.683413 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:40.683562 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:40.684339 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.685778 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.687298 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.687989 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.688099 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.690667 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:40.690671 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:40.690671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:40.690682 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:40.692768 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:40.693281 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:40.693285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:40.693285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:40.693406 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:40.693667 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:40.693668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:40.693668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:40.693951 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:40.694039 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:40.694289 (+ 250us) server_negotiation.cc:299] Negotiation successful
0925 13:09:40.694414 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":76,"threads_started":1}
I20260925 13:09:40.695253 15300 heartbeater.cc:347] Connected to a master server at 127.0.206.126:44859
I20260925 13:09:40.695360 15300 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:40.695523 15300 heartbeater.cc:511] Master 127.0.206.126:44859 requested a full tablet report, sending...
I20260925 13:09:40.696195 15112 ts_manager.cc:194] Registered new tserver with Master: 03a1a95e4d5b4a89b0d3d80a5c9743e4 (127.0.206.65:41397)
I20260925 13:09:40.697353 15112 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:43545
I20260925 13:09:40.698189 15300 heartbeater.cc:503] Master 127.0.206.126:44859 was elected leader, sending a full tablet report...
Sep 25 13:09:40 dist-test-slave-chkf krb5kdc[14535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260925 13:09:40.704774 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.696721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:41397 (local address 127.0.0.1:37964)
0925 13:09:40.697052 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:40.697082 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:40.697137 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:40.697476 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:40.697480 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:40.697873 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:40.697918 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.697920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.699056 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.699056 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.699601 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.699603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.700068 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.700069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.700153 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.700598 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:40.700603 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:40.701799 (+ 1196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:40.703598 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:40.703604 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:40.703606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:40.703826 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:40.704051 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:40.704051 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:40.704052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:40.704113 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:40.704421 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:40.704427 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:40.704472 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 13:09:40.704563 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 13:09:40.704666 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":68,"threads_started":1}
I20260925 13:09:40.704806 15308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.696921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37964 (local address 127.0.206.65:41397)
0925 13:09:40.697244 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:40.697248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:40.697257 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:09:40.697285 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:40.697287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:40.697300 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:40.697357 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:40.698005 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.698969 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.699700 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.699979 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.700064 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.701894 (+ 1830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:40.701900 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:40.701901 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:40.701907 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:40.703506 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:40.703918 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:40.703921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:40.703921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:40.703976 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:40.704173 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:40.704174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:40.704174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:40.704343 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:40.704424 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:40.704596 (+ 172us) server_negotiation.cc:299] Negotiation successful
0925 13:09:40.704676 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":60,"threads_started":1}
I20260925 13:09:40.705523 15218 tablet_service.cc:1514] Processing DeleteTablet for tablet 12f326eb910f4826a65fa7155d0f5c77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:09:40 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:37964
I20260925 13:09:40.705928 15309 tablet_replica.cc:330] stopping tablet replica
I20260925 13:09:40.706043 15309 raft_consensus.cc:2229] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:09:40.706103 15309 raft_consensus.cc:2258] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:09:40.706364 15309 ts_tablet_manager.cc:1905] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:09:40.708184 15309 ts_tablet_manager.cc:1918] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:09:40.708272 15309 log.cc:1196] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/wal/wals/12f326eb910f4826a65fa7155d0f5c77
I20260925 13:09:40.708500 15309 ts_tablet_manager.cc:1939] T 12f326eb910f4826a65fa7155d0f5c77 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Deleting consensus metadata
I20260925 13:09:40.708933 15099 catalog_manager.cc:4841] TS 03a1a95e4d5b4a89b0d3d80a5c9743e4 (127.0.206.65:41397): tablet 12f326eb910f4826a65fa7155d0f5c77 (table test-table [id=f2bb2161989146ee92a23594b89ba3ac]) successfully deleted
W20260925 13:09:40.781100 15306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:40.781276 15306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:40.781298 15306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:40.782452 15306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:40.782491 15306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:40.782517 15306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:09:40.783933 15306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:34897
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=45895
--tserver_master_addrs=127.0.206.126:44859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:40.784116 15306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:40.784339 15306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:40.789096 15314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:40.789130 15317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:40.789080 15315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:40.789353 15306 server_base.cc:1034] running on GCE node
I20260925 13:09:40.789559 15306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:40.789788 15306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:40.790920 15306 hybrid_clock.cc:648] HybridClock initialized: now 1790341780790904 us; error 35 us; skew 500 ppm
Sep 25 13:09:40 dist-test-slave-chkf krb5kdc[14535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:40.793154 15306 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:09:40.793542 15306 webserver.cc:460] Webserver started at http://127.0.206.66:45895/ using document root <none> and password file <none>
I20260925 13:09:40.793753 15306 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:40.793807 15306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:40.795099 15306 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:09:40.795655 15324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:40.795828 15306 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:09:40.795914 15306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc"
format_stamp: "Formatted at 2026-09-25 13:09:39 on dist-test-slave-chkf"
server_key: "c20f391f7c1dd75081623e9d8cfeae37"
server_key_iv: "742273ebcd8efa40f16e8daf1ada1ecc"
server_key_version: "encryptionkey@0"
I20260925 13:09:40.796197 15306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:40.819907 15306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:40.821545 15306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:40.821693 15306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:40.821915 15306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:09:40.822193 15306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:09:40.822233 15306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:40.822273 15306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:09:40.822299 15306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:40.827913 15306 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:34897
I20260925 13:09:40.827970 15437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:34897 every 8 connection(s)
I20260925 13:09:40.828305 15306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
Sep 25 13:09:40 dist-test-slave-chkf krb5kdc[14535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:40.836514 15303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.828999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:41203 (local address 127.0.206.126:44859)
0925 13:09:40.829039 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:40.829041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:40.829240 (+ 199us) server_negotiation.cc:407] Connection header received
0925 13:09:40.829672 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:40.829673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:40.829684 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:40.829746 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:40.830572 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.831167 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.832123 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.832358 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.832409 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.834270 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:40.834272 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:40.834272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:40.834279 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:40.835480 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:40.835835 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:40.835836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:40.835836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:40.835884 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:40.836052 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:40.836052 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:40.836052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:40.836153 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:40.836199 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:40.836395 (+ 196us) server_negotiation.cc:299] Negotiation successful
0925 13:09:40.836455 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 13:09:40.836660 15440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.828975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:41203)
0925 13:09:40.829121 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:40.829159 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:40.829597 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:40.829794 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:40.829797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:40.830324 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:40.830497 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.830501 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.831269 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.831270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.832049 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.832051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.832428 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.832429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.832522 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.832918 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:40.832925 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:40.834181 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:40.835547 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:40.835553 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:40.835554 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:40.835771 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:40.835926 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:40.835926 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:40.835927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:40.836004 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:40.836207 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:40.836211 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:40.836310 (+ 99us) client_negotiation.cc:769] Sending connection context
0925 13:09:40.836384 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 13:09:40.836486 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260925 13:09:40.837143 15438 heartbeater.cc:347] Connected to a master server at 127.0.206.126:44859
I20260925 13:09:40.837214 15438 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:40.837395 15438 heartbeater.cc:511] Master 127.0.206.126:44859 requested a full tablet report, sending...
I20260925 13:09:40.837767 15112 ts_manager.cc:194] Registered new tserver with Master: 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66:34897)
I20260925 13:09:40.838117 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 15306
I20260925 13:09:40.838269 15112 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:41203
I20260925 13:09:40.838424 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:41463
--local_ip_for_outbound_sockets=127.0.206.67
--tserver_master_addrs=127.0.206.126:44859
--webserver_port=45473
--webserver_interface=127.0.206.67
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:40.915933 15441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:40.916108 15441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:40.916131 15441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:40.917817 15441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:40.917860 15441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:40.917886 15441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:09:40.919325 15441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:41463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=45473
--tserver_master_addrs=127.0.206.126:44859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:40.919548 15441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:40.919780 15441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:40.924450 15446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:40.924466 15447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:40.924515 15441 server_base.cc:1034] running on GCE node
W20260925 13:09:40.924467 15449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:40.924839 15441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:40.925086 15441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:40.926265 15441 hybrid_clock.cc:648] HybridClock initialized: now 1790341780926250 us; error 27 us; skew 500 ppm
Sep 25 13:09:40 dist-test-slave-chkf krb5kdc[14535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:40.928591 15441 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:09:40.928979 15441 webserver.cc:460] Webserver started at http://127.0.206.67:45473/ using document root <none> and password file <none>
I20260925 13:09:40.929201 15441 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:40.929278 15441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:40.930495 15441 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:09:40.931059 15456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:40.931205 15441 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:40.931293 15441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "034c6426204c4846a2ea10f6233bca00"
format_stamp: "Formatted at 2026-09-25 13:09:40 on dist-test-slave-chkf"
server_key: "c4efb0230a15fcb4099ca95c99a927d4"
server_key_iv: "444d4a71286b85463ade0757f4bd76a4"
server_key_version: "encryptionkey@0"
I20260925 13:09:40.931615 15441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:40.939571 15441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:40.941210 15441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:40.941315 15441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:40.941462 15441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:09:40.941670 15441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:09:40.941697 15441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:40.941716 15441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:09:40.941728 15441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:40.947062 15441 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:41463
I20260925 13:09:40.947129 15569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:41463 every 8 connection(s)
I20260925 13:09:40.947351 15441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
Sep 25 13:09:40 dist-test-slave-chkf krb5kdc[14535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:40.952963 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 15441
I20260925 13:09:40.955883 15303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.947980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:36751 (local address 127.0.206.126:44859)
0925 13:09:40.948037 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:40.948039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:40.948195 (+ 156us) server_negotiation.cc:407] Connection header received
0925 13:09:40.948614 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:40.948615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:40.948625 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:40.948674 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:40.949463 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.949938 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.950892 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.951072 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.951130 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.953026 (+ 1896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:40.953028 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:40.953028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:40.953035 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:40.954517 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:40.954992 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:40.954993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:40.954993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:40.955042 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:40.955307 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:40.955308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:40.955308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:40.955427 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:40.955500 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:40.955712 (+ 212us) server_negotiation.cc:299] Negotiation successful
0925 13:09:40.955792 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 13:09:40.955940 15572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.947957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:36751)
0925 13:09:40.948105 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:40.948136 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:40.948532 (+ 396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:40.948718 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:40.948720 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:40.949226 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:40.949393 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.949397 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.950009 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.950009 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.950826 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.950828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.951148 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.951149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.951243 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.951629 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:40.951637 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:40.952931 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:40.954589 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:40.954597 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:40.954599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:40.954919 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:40.955108 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:40.955109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:40.955109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:40.955237 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:40.955490 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:40.955494 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:40.955592 (+ 98us) client_negotiation.cc:769] Sending connection context
0925 13:09:40.955666 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 13:09:40.955778 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":90,"threads_started":1}
I20260925 13:09:40.956384 15570 heartbeater.cc:347] Connected to a master server at 127.0.206.126:44859
I20260925 13:09:40.956475 15570 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:40.956653 15570 heartbeater.cc:511] Master 127.0.206.126:44859 requested a full tablet report, sending...
I20260925 13:09:40.957012 15112 ts_manager.cc:194] Registered new tserver with Master: 034c6426204c4846a2ea10f6233bca00 (127.0.206.67:41463)
I20260925 13:09:40.957322 15112 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:36751
I20260925 13:09:40.960026 15574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.953525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38870 (local address 127.0.206.126:44859)
0925 13:09:40.953657 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:40.953658 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:40.953664 (+ 6us) server_negotiation.cc:407] Connection header received
0925 13:09:40.953681 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:40.953681 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:40.953691 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:40.953770 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:40.954436 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.955195 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.956213 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.956477 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.956546 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.957745 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:40.957747 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:40.957747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:40.957753 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:40.959110 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:40.959457 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:40.959458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:40.959458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:40.959518 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:40.959660 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:40.959660 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:40.959660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:40.959741 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:40.959839 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:40.959890 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 13:09:40.959944 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
I20260925 13:09:40.960652 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:09:40.963006 15574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.960965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38880 (local address 127.0.206.126:44859)
0925 13:09:40.961006 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:40.961007 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:40.961089 (+ 82us) server_negotiation.cc:407] Connection header received
0925 13:09:40.961144 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:40.961145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:40.961167 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:40.961223 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:09:40.961288 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.961690 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.962349 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.962516 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.962561 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.962675 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:09:40.962818 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:09:40.962874 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:40.962909 (+ 35us) server_negotiation.cc:299] Negotiation successful
0925 13:09:40.962924 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 13:09:40.963274 15112 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:38880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:09:40.964751 15112 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:09:40.966607 15218 tablet_service.cc:1467] Processing CreateTablet for tablet b15bf61c24bb4c679d1a7f19c99f3273 (DEFAULT_TABLE table=test-table [id=fe99b85a8df542d59d78bf5767b630f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:09:40.966811 15218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b15bf61c24bb4c679d1a7f19c99f3273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:40.968276 15192 tablet_bootstrap.cc:492] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Bootstrap starting.
I20260925 13:09:40.968587 15192 tablet_bootstrap.cc:654] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:40.970193 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.966313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:34897 (local address 127.0.0.1:33378)
0925 13:09:40.966342 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:40.966352 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:40.966387 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:40.966785 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:40.966786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:40.966788 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:40.966824 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.966826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.968059 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.968060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.969004 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.969005 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.969870 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.969871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.969945 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.969997 (+ 52us) client_negotiation.cc:769] Sending connection context
0925 13:09:40.970068 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 13:09:40.970119 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16,"mutex_wait_us":37}
I20260925 13:09:40.970201 15192 tablet_bootstrap.cc:492] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: No bootstrap required, opened a new log
I20260925 13:09:40.970221 15577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.966420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33378 (local address 127.0.206.66:34897)
0925 13:09:40.966582 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:40.966587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:40.966597 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:09:40.966634 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:40.966636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:40.966650 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:40.966900 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:40.966964 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.967963 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.969084 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.969787 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.969872 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.969924 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:40.970101 (+ 177us) server_negotiation.cc:299] Negotiation successful
0925 13:09:40.970143 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260925 13:09:40.970253 15192 ts_tablet_manager.cc:1397] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 13:09:40.970407 15192 raft_consensus.cc:348] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } } peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } }
I20260925 13:09:40.970474 15192 raft_consensus.cc:374] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:40.970512 15192 raft_consensus.cc:729] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a1a95e4d5b4a89b0d3d80a5c9743e4, State: Initialized, Role: FOLLOWER
I20260925 13:09:40.970566 15192 consensus_queue.cc:260] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } } peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } }
I20260925 13:09:40.970657 15192 ts_tablet_manager.cc:1428] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:40.970959 15372 tablet_service.cc:1467] Processing CreateTablet for tablet b15bf61c24bb4c679d1a7f19c99f3273 (DEFAULT_TABLE table=test-table [id=fe99b85a8df542d59d78bf5767b630f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:09:40.971269 15372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b15bf61c24bb4c679d1a7f19c99f3273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:40.972164 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.966376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:41463 (local address 127.0.0.1:52270)
0925 13:09:40.968325 (+ 1949us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:40.968336 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:40.968392 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:40.968597 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:40.968598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:40.968600 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:40.968688 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.968690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.969832 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.969833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.970866 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:40.970869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.971869 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.971870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:40.971932 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.971934 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:09:40.972058 (+ 124us) client_negotiation.cc:240] Negotiation successful
0925 13:09:40.972097 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1933,"thread_start_us":87,"threads_started":1}
I20260925 13:09:40.972231 15578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:40.966587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52270 (local address 127.0.206.67:41463)
0925 13:09:40.966786 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:40.966790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:40.968454 (+ 1664us) server_negotiation.cc:407] Connection header received
0925 13:09:40.968491 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:40.968493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:40.968508 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:40.968585 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:40.968787 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.969742 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.970967 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:40.971652 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:40.971698 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:40.971747 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:40.972093 (+ 346us) server_negotiation.cc:299] Negotiation successful
0925 13:09:40.972139 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":76,"threads_started":1}
I20260925 13:09:40.972801 15504 tablet_service.cc:1467] Processing CreateTablet for tablet b15bf61c24bb4c679d1a7f19c99f3273 (DEFAULT_TABLE table=test-table [id=fe99b85a8df542d59d78bf5767b630f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:09:40.973079 15504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b15bf61c24bb4c679d1a7f19c99f3273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:40.974340 15580 tablet_bootstrap.cc:492] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: Bootstrap starting.
I20260925 13:09:40.975025 15580 tablet_bootstrap.cc:654] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:40.975554 15580 log.cc:824] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:40.976086 15582 tablet_bootstrap.cc:492] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00: Bootstrap starting.
I20260925 13:09:40.976287 15580 tablet_bootstrap.cc:492] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: No bootstrap required, opened a new log
I20260925 13:09:40.976351 15580 ts_tablet_manager.cc:1397] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:09:40.976666 15582 tablet_bootstrap.cc:654] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:40.977105 15582 log.cc:824] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:40.977761 15582 tablet_bootstrap.cc:492] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00: No bootstrap required, opened a new log
I20260925 13:09:40.977837 15582 ts_tablet_manager.cc:1397] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:09:40.977957 15580 raft_consensus.cc:348] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } } peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } }
I20260925 13:09:40.978142 15580 raft_consensus.cc:374] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:40.978188 15580 raft_consensus.cc:729] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a9f8a96e5aa4c89b0a6242e211f8dfc, State: Initialized, Role: FOLLOWER
I20260925 13:09:40.978318 15580 consensus_queue.cc:260] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } } peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } }
I20260925 13:09:40.978598 15580 ts_tablet_manager.cc:1428] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 13:09:40.978611 15438 heartbeater.cc:503] Master 127.0.206.126:44859 was elected leader, sending a full tablet report...
I20260925 13:09:40.979784 15582 raft_consensus.cc:348] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } } peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } }
I20260925 13:09:40.979903 15582 raft_consensus.cc:374] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:40.979924 15582 raft_consensus.cc:729] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 034c6426204c4846a2ea10f6233bca00, State: Initialized, Role: FOLLOWER
I20260925 13:09:40.980020 15582 consensus_queue.cc:260] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } } peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } }
I20260925 13:09:40.980275 15582 ts_tablet_manager.cc:1428] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 13:09:40.980284 15570 heartbeater.cc:503] Master 127.0.206.126:44859 was elected leader, sending a full tablet report...
W20260925 13:09:41.078871 15439 tablet.cc:2369] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:09:41.183374 15301 tablet.cc:2369] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:09:41.197808 15571 tablet.cc:2369] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:09:41.376818 15584 raft_consensus.cc:482] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:09:41.376940 15584 raft_consensus.cc:504] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } } peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } }
I20260925 13:09:41.377382 15584 leader_election.cc:288] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 034c6426204c4846a2ea10f6233bca00 (127.0.206.67:41463), 03a1a95e4d5b4a89b0d3d80a5c9743e4 (127.0.206.65:41397)
I20260925 13:09:41.380297 15578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:41.377654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:34229 (local address 127.0.206.67:41463)
0925 13:09:41.377720 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:41.377722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:41.377779 (+ 57us) server_negotiation.cc:407] Connection header received
0925 13:09:41.377848 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:41.377849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:41.377860 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:41.377935 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:41.378093 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:41.378564 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:41.379560 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:41.379932 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:41.380017 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:41.380033 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:41.380184 (+ 151us) server_negotiation.cc:299] Negotiation successful
0925 13:09:41.380219 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 13:09:41.380321 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:41.377616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:57369 (local address 127.0.206.65:41397)
0925 13:09:41.377782 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:41.377784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:41.377792 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:09:41.377820 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:41.377821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:41.377855 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:41.377942 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:41.378092 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:41.378567 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:41.379555 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:41.379984 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:41.380072 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:41.380150 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:41.380211 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 13:09:41.380250 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":43,"threads_started":1}
I20260925 13:09:41.380344 15587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:41.377554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:41397 (local address 127.0.206.66:57369)
0925 13:09:41.377676 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:41.377688 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:41.377748 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:41.377945 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:41.377946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:41.377948 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:41.378021 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:41.378023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:41.378652 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:41.378653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:41.379461 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:41.379463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:41.380078 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:41.380079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:41.380143 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:41.380147 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:09:41.380208 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 13:09:41.380247 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":28,"thread_start_us":56,"threads_started":1}
I20260925 13:09:41.380347 15586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:41.377567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:41463 (local address 127.0.206.66:34229)
0925 13:09:41.377690 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:41.377700 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:41.377790 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:41.377939 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:41.377940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:41.377942 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:41.378016 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:41.378018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:41.378645 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:41.378646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:41.379457 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:41.379459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:41.380015 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:41.380016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:41.380090 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:41.380093 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:09:41.380177 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 13:09:41.380216 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260925 13:09:41.380627 15243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15bf61c24bb4c679d1a7f19c99f3273" candidate_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" is_pre_election: true
I20260925 13:09:41.380816 15243 raft_consensus.cc:2454] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a9f8a96e5aa4c89b0a6242e211f8dfc in term 0.
I20260925 13:09:41.380906 15524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15bf61c24bb4c679d1a7f19c99f3273" candidate_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "034c6426204c4846a2ea10f6233bca00" is_pre_election: true
I20260925 13:09:41.381049 15328 leader_election.cc:302] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a1a95e4d5b4a89b0d3d80a5c9743e4, 6a9f8a96e5aa4c89b0a6242e211f8dfc; no voters:
I20260925 13:09:41.381063 15524 raft_consensus.cc:2454] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a9f8a96e5aa4c89b0a6242e211f8dfc in term 0.
I20260925 13:09:41.381181 15584 raft_consensus.cc:2781] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:09:41.381256 15584 raft_consensus.cc:482] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:09:41.381281 15584 raft_consensus.cc:3037] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:41.381987 15584 raft_consensus.cc:504] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } } peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } }
I20260925 13:09:41.382115 15584 leader_election.cc:288] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [CANDIDATE]: Term 1 election: Requested vote from peers 034c6426204c4846a2ea10f6233bca00 (127.0.206.67:41463), 03a1a95e4d5b4a89b0d3d80a5c9743e4 (127.0.206.65:41397)
I20260925 13:09:41.382294 15524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15bf61c24bb4c679d1a7f19c99f3273" candidate_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "034c6426204c4846a2ea10f6233bca00"
I20260925 13:09:41.382290 15243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b15bf61c24bb4c679d1a7f19c99f3273" candidate_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4"
I20260925 13:09:41.382377 15243 raft_consensus.cc:3037] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:41.382378 15524 raft_consensus.cc:3037] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:41.383261 15524 raft_consensus.cc:2454] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a9f8a96e5aa4c89b0a6242e211f8dfc in term 1.
I20260925 13:09:41.383261 15243 raft_consensus.cc:2454] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a9f8a96e5aa4c89b0a6242e211f8dfc in term 1.
I20260925 13:09:41.383443 15326 leader_election.cc:302] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 034c6426204c4846a2ea10f6233bca00, 6a9f8a96e5aa4c89b0a6242e211f8dfc; no voters:
I20260925 13:09:41.383539 15584 raft_consensus.cc:2781] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:09:41.383740 15584 raft_consensus.cc:686] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 1 LEADER]: Becoming Leader. State: Replica: 6a9f8a96e5aa4c89b0a6242e211f8dfc, State: Running, Role: LEADER
I20260925 13:09:41.383819 15584 consensus_queue.cc:237] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } } peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } }
I20260925 13:09:41.384552 15112 catalog_manager.cc:5495] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66). New cstate: current_term: 1 leader_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } health_report { overall_health: UNKNOWN } } }
I20260925 13:09:41.409291 15577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:41.407267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33390 (local address 127.0.206.66:34897)
0925 13:09:41.407303 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:41.407306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:41.407320 (+ 14us) server_negotiation.cc:407] Connection header received
0925 13:09:41.407366 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:41.407366 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:41.407381 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:41.407432 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:09:41.407546 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:41.407980 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:41.408687 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:41.408861 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:41.408909 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:41.409036 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:09:41.409124 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:09:41.409181 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:41.409220 (+ 39us) server_negotiation.cc:299] Negotiation successful
0925 13:09:41.409240 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:09:41.410959 15243 raft_consensus.cc:1261] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Refusing update from remote peer 6a9f8a96e5aa4c89b0a6242e211f8dfc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:09:41.410981 15524 raft_consensus.cc:1261] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00 [term 1 FOLLOWER]: Refusing update from remote peer 6a9f8a96e5aa4c89b0a6242e211f8dfc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:09:41.411295 15589 consensus_queue.cc:1035] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [LEADER]: Connected to new peer: Peer: permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:09:41.411509 15584 consensus_queue.cc:1035] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [LEADER]: Connected to new peer: Peer: permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:09:41.413101 15592 mvcc.cc:205] Tried to move back new op lower bound from 7333239936656879616 to 7333239936548577280. Current Snapshot: MvccSnapshot[applied={T|T < 7333239936656879616}]
I20260925 13:09:41.413726 15593 mvcc.cc:205] Tried to move back new op lower bound from 7333239936656879616 to 7333239936548577280. Current Snapshot: MvccSnapshot[applied={T|T < 7333239936656879616 or (T in {7333239936656879616})}]
I20260925 13:09:41.414407 15595 mvcc.cc:205] Tried to move back new op lower bound from 7333239936656879616 to 7333239936548577280. Current Snapshot: MvccSnapshot[applied={T|T < 7333239936656879616}]
I20260925 13:09:41.415498 15112 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:38880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:09:41.415582 15112 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:38880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:09:41.416276 15112 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377: Sending DeleteTablet for 3 replicas of tablet b15bf61c24bb4c679d1a7f19c99f3273
I20260925 13:09:41.416572 15218 tablet_service.cc:1514] Processing DeleteTablet for tablet b15bf61c24bb4c679d1a7f19c99f3273 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:09:41 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:37964
I20260925 13:09:41.416579 15372 tablet_service.cc:1514] Processing DeleteTablet for tablet b15bf61c24bb4c679d1a7f19c99f3273 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:09:41 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:33378
I20260925 13:09:41.416610 15504 tablet_service.cc:1514] Processing DeleteTablet for tablet b15bf61c24bb4c679d1a7f19c99f3273 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:09:41 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:52270
I20260925 13:09:41.416802 15602 tablet_replica.cc:330] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: stopping tablet replica
I20260925 13:09:41.416885 15602 raft_consensus.cc:2229] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:09:41.416882 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 15150
I20260925 13:09:41.416986 15602 raft_consensus.cc:2258] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:09:41.417919 15602 ts_tablet_manager.cc:1905] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:09:41.419871 15602 ts_tablet_manager.cc:1918] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:09:41.419931 15602 log.cc:1196] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/b15bf61c24bb4c679d1a7f19c99f3273
I20260925 13:09:41.420194 15602 ts_tablet_manager.cc:1939] T b15bf61c24bb4c679d1a7f19c99f3273 P 6a9f8a96e5aa4c89b0a6242e211f8dfc: Deleting consensus metadata
I20260925 13:09:41.420717 15099 catalog_manager.cc:4841] TS 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66:34897): tablet b15bf61c24bb4c679d1a7f19c99f3273 (table test-table [id=fe99b85a8df542d59d78bf5767b630f6]) successfully deleted
I20260925 13:09:41.422844 15603 tablet_replica.cc:330] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00: stopping tablet replica
I20260925 13:09:41.422930 15603 raft_consensus.cc:2229] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:09:41.423010 15603 raft_consensus.cc:2258] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:09:41.423398 15603 ts_tablet_manager.cc:1905] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:09:41.423606 15099 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:41397 (error 108)
I20260925 13:09:41.424028 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 15306
I20260925 13:09:41.424160 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:41.424047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:41397 (local address 127.0.0.1:37976)
0925 13:09:41.424089 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:41.424123 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:41397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260925 13:09:41.424263 15099 catalog_manager.cc:4584] TS 03a1a95e4d5b4a89b0d3d80a5c9743e4 (127.0.206.65:41397): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b15bf61c24bb4c679d1a7f19c99f3273: Network error: Client connection negotiation failed: client connection to 127.0.206.65:41397: connect: Connection refused (error 111)
I20260925 13:09:41.427639 15603 ts_tablet_manager.cc:1918] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:09:41.427732 15603 log.cc:1196] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/b15bf61c24bb4c679d1a7f19c99f3273
I20260925 13:09:41.427994 15603 ts_tablet_manager.cc:1939] T b15bf61c24bb4c679d1a7f19c99f3273 P 034c6426204c4846a2ea10f6233bca00: Deleting consensus metadata
I20260925 13:09:41.428381 15097 catalog_manager.cc:4841] TS 034c6426204c4846a2ea10f6233bca00 (127.0.206.67:41463): tablet b15bf61c24bb4c679d1a7f19c99f3273 (table test-table [id=fe99b85a8df542d59d78bf5767b630f6]) successfully deleted
I20260925 13:09:41.428848 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 15441
I20260925 13:09:41.433997 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 15079
I20260925 13:09:41.439323 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:44859
--webserver_interface=127.0.206.126
--webserver_port=38369
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:44859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:41.519454 15605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:41.519644 15605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:41.519666 15605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:41.520769 15605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:09:41.520808 15605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:41.520823 15605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:41.520834 15605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:09:41.520888 15605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:09:41.522269 15605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:44859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:44859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=38369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:41.522503 15605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:41.522744 15605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:41.527503 15610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:41.527464 15611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:41.527632 15613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:41.527863 15605 server_base.cc:1034] running on GCE node
I20260925 13:09:41.528079 15605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:41.528357 15605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:41.529522 15605 hybrid_clock.cc:648] HybridClock initialized: now 1790341781529510 us; error 22 us; skew 500 ppm
Sep 25 13:09:41 dist-test-slave-chkf krb5kdc[14535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:41.531831 15605 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:09:41.532217 15605 webserver.cc:460] Webserver started at http://127.0.206.126:38369/ using document root <none> and password file <none>
I20260925 13:09:41.532428 15605 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:41.532491 15605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:41.533774 15605 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:09:41.534440 15620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:41.534619 15605 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:09:41.534757 15605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "79b82f96623b497483133dde7004c377"
format_stamp: "Formatted at 2026-09-25 13:09:39 on dist-test-slave-chkf"
server_key: "28ea354e848d755fa01fedb1e1b512e1"
server_key_iv: "ac7c2e601abbeda19fe6f1988edb6d4a"
server_key_version: "encryptionkey@0"
I20260925 13:09:41.535079 15605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:41.559624 15605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:41.561395 15605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:41.561547 15605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:41.565527 15605 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:44859
I20260925 13:09:41.565565 15672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:44859 every 8 connection(s)
I20260925 13:09:41.565944 15605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:09:41.567430 15673 sys_catalog.cc:263] Verifying existing consensus state
I20260925 13:09:41.568069 15673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377: Bootstrap starting.
I20260925 13:09:41.574479 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 15605
I20260925 13:09:41.574762 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:41397
--local_ip_for_outbound_sockets=127.0.206.65
--tserver_master_addrs=127.0.206.126:44859
--webserver_port=38629
--webserver_interface=127.0.206.65
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:09:41.577049 15673 log.cc:824] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:41.580108 15673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:09:41.580449 15673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377: Bootstrap complete.
I20260925 13:09:41.584329 15673 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b82f96623b497483133dde7004c377" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44859 } }
I20260925 13:09:41.584681 15673 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b82f96623b497483133dde7004c377, State: Initialized, Role: FOLLOWER
I20260925 13:09:41.584882 15673 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b82f96623b497483133dde7004c377" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44859 } }
I20260925 13:09:41.584969 15673 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:09:41.585022 15673 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:09:41.585074 15673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:09:41.586535 15673 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b82f96623b497483133dde7004c377" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44859 } }
I20260925 13:09:41.586639 15673 leader_election.cc:302] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79b82f96623b497483133dde7004c377; no voters:
I20260925 13:09:41.586798 15673 leader_election.cc:288] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 13:09:41.586861 15677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 13:09:41.587006 15677 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [term 3 LEADER]: Becoming Leader. State: Replica: 79b82f96623b497483133dde7004c377, State: Running, Role: LEADER
I20260925 13:09:41.587119 15677 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b82f96623b497483133dde7004c377" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44859 } }
I20260925 13:09:41.587014 15673 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:09:41.587805 15679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79b82f96623b497483133dde7004c377. Latest consensus state: current_term: 3 leader_uuid: "79b82f96623b497483133dde7004c377" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b82f96623b497483133dde7004c377" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44859 } } }
I20260925 13:09:41.587926 15679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [sys.catalog]: This master's current role is: LEADER
I20260925 13:09:41.587807 15678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "79b82f96623b497483133dde7004c377" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79b82f96623b497483133dde7004c377" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44859 } } }
I20260925 13:09:41.588035 15678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79b82f96623b497483133dde7004c377 [sys.catalog]: This master's current role is: LEADER
I20260925 13:09:41.588127 15687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:09:41.588876 15687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:09:41.588982 15687 catalog_manager.cc:1259] Loaded cluster ID: 74e0400851224824942d3019ca2a06ea
I20260925 13:09:41.589040 15687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260925 13:09:41.589473 15687 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fed2e5c19ad google::LogMessage::Fail() at ??:0
@ 0x7fed2e5c488a google::LogMessage::SendToLog() at ??:0
@ 0x7fed2e5c14de google::LogMessage::Flush() at ??:0
@ 0x7fed2e5c2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fed364e20f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fed364eb8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fed3048f0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fed30484954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fed312e86db start_thread at ??:0
@ 0x7fed2d0bf61f clone at ??:0
W20260925 13:09:41.667280 15675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:41.667481 15675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:41.667532 15675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:41.668645 15675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:41.668715 15675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:41.668776 15675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:09:41.670128 15675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:41397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=38629
--tserver_master_addrs=127.0.206.126:44859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:41.670372 15675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:41.670590 15675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:41.676621 15712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:41.676596 15710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:41.676606 15709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:41.676877 15675 server_base.cc:1034] running on GCE node
I20260925 13:09:41.677086 15675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:41.677345 15675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:41.678500 15675 hybrid_clock.cc:648] HybridClock initialized: now 1790341781678478 us; error 37 us; skew 500 ppm
Sep 25 13:09:41 dist-test-slave-chkf krb5kdc[14535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:41.680858 15675 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:09:41.681291 15675 webserver.cc:460] Webserver started at http://127.0.206.65:38629/ using document root <none> and password file <none>
I20260925 13:09:41.681509 15675 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:41.681576 15675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:41.682960 15675 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:09:41.683658 15719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:41.683863 15675 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:09:41.683947 15675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4"
format_stamp: "Formatted at 2026-09-25 13:09:39 on dist-test-slave-chkf"
server_key: "2b13c364abf63466b3b1ffcbb4441112"
server_key_iv: "2530d81ec86a82cd80d8f181709c3dd7"
server_key_version: "encryptionkey@0"
I20260925 13:09:41.684255 15675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:41.693802 15675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:41.695924 15675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:41.696070 15675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:41.696278 15675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:09:41.696674 15726 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 13:09:41.698613 15675 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 13:09:41.698680 15675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 13:09:41.698745 15675 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 13:09:41.699342 15675 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 13:09:41.699394 15675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 13:09:41.699474 15726 tablet_bootstrap.cc:492] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Bootstrap starting.
I20260925 13:09:41.706295 15675 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:41397
I20260925 13:09:41.706377 15833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:41397 every 8 connection(s)
I20260925 13:09:41.706673 15675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:09:41.710300 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 15675
I20260925 13:09:41.710671 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:34897
--local_ip_for_outbound_sockets=127.0.206.66
--tserver_master_addrs=127.0.206.126:44859
--webserver_port=45895
--webserver_interface=127.0.206.66
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:09:41.710894 15726 log.cc:824] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:41.713541 15726 tablet_bootstrap.cc:492] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:09:41.713908 15726 tablet_bootstrap.cc:492] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Bootstrap complete.
I20260925 13:09:41.715335 15726 ts_tablet_manager.cc:1397] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.015s
I20260925 13:09:41.716769 15726 raft_consensus.cc:348] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } } peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } }
I20260925 13:09:41.717146 15726 raft_consensus.cc:729] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a1a95e4d5b4a89b0d3d80a5c9743e4, State: Initialized, Role: FOLLOWER
I20260925 13:09:41.717335 15726 consensus_queue.cc:260] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } } peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } }
I20260925 13:09:41.717641 15726 ts_tablet_manager.cc:1428] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260925 13:09:41.790191 15838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:41.790407 15838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:41.790443 15838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:41.791626 15838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:41.791692 15838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:41.791734 15838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:09:41.793697 15838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:34897
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=45895
--tserver_master_addrs=127.0.206.126:44859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:41.793926 15838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:41.794158 15838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:41.796315 15836 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:09:41.707319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:45011)
0925 13:09:41.707468 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:41.707505 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:41.707921 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:41.796146 (+ 88225us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
W20260925 13:09:41.796526 15834 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:44859 (0 consecutive failures): Network error: Failed to ping master at 127.0.206.126:44859: Client connection negotiation failed: client connection to 127.0.206.126:44859: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 13:09:41.796885 15836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:41.796746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:41111)
0925 13:09:41.796806 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:41.796856 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 13:09:41.797253 15836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:41.797163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:55999)
0925 13:09:41.797208 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:41.797235 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 13:09:41.797381 15834 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 13:09:41.799925 15847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:41.800014 15838 server_base.cc:1034] running on GCE node
W20260925 13:09:41.799948 15844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:41.799921 15845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:41.800276 15838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:41.800459 15838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:41.801621 15838 hybrid_clock.cc:648] HybridClock initialized: now 1790341781801609 us; error 28 us; skew 500 ppm
Sep 25 13:09:41 dist-test-slave-chkf krb5kdc[14535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:41.804219 15838 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:09:41.804643 15838 webserver.cc:460] Webserver started at http://127.0.206.66:45895/ using document root <none> and password file <none>
I20260925 13:09:41.804850 15838 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:41.804891 15838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:41.806124 15838 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:09:41.806663 15854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:41.806829 15838 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:41.806923 15838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc"
format_stamp: "Formatted at 2026-09-25 13:09:39 on dist-test-slave-chkf"
server_key: "c20f391f7c1dd75081623e9d8cfeae37"
server_key_iv: "742273ebcd8efa40f16e8daf1ada1ecc"
server_key_version: "encryptionkey@0"
I20260925 13:09:41.807237 15838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:41.825901 15838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:41.828172 15838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:41.828330 15838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:41.828540 15838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:09:41.828831 15838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:09:41.828887 15838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:41.828927 15838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:09:41.828958 15838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:41.834605 15838 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:34897
I20260925 13:09:41.834651 15967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:34897 every 8 connection(s)
I20260925 13:09:41.834971 15838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:09:41.835743 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 15838
I20260925 13:09:41.836079 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:41.835568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:58843)
0925 13:09:41.835799 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:41.835888 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260925 13:09:41.836066 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:41463
--local_ip_for_outbound_sockets=127.0.206.67
--tserver_master_addrs=127.0.206.126:44859
--webserver_port=45473
--webserver_interface=127.0.206.67
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:41.836500 15968 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:44859 (0 consecutive failures): Network error: Failed to ping master at 127.0.206.126:44859: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
I20260925 13:09:41.836767 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:41.836673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:54059)
0925 13:09:41.836719 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:41.836747 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 13:09:41.837085 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:41.837006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:48555)
0925 13:09:41.837046 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:41.837068 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 13:09:41.837301 15968 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 13:09:41.914649 15971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:41.914897 15971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:41.914920 15971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:41.916039 15971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:41.916077 15971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:41.916102 15971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:09:41.917547 15971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:41463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=45473
--tserver_master_addrs=127.0.206.126:44859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:41.917768 15971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:41.917976 15971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:41.922735 15977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:41.922756 15976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:41.922798 15971 server_base.cc:1034] running on GCE node
W20260925 13:09:41.922803 15979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:41.923125 15971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:41.923352 15971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:41.924518 15971 hybrid_clock.cc:648] HybridClock initialized: now 1790341781924504 us; error 48 us; skew 500 ppm
Sep 25 13:09:41 dist-test-slave-chkf krb5kdc[14535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:41.926844 15971 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:09:41.927250 15971 webserver.cc:460] Webserver started at http://127.0.206.67:45473/ using document root <none> and password file <none>
I20260925 13:09:41.927469 15971 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:41.927546 15971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:41.928791 15971 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:09:41.929337 15986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:41.929502 15971 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:09:41.929593 15971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "034c6426204c4846a2ea10f6233bca00"
format_stamp: "Formatted at 2026-09-25 13:09:40 on dist-test-slave-chkf"
server_key: "c4efb0230a15fcb4099ca95c99a927d4"
server_key_iv: "444d4a71286b85463ade0757f4bd76a4"
server_key_version: "encryptionkey@0"
I20260925 13:09:41.929934 15971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:41.937825 15971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:41.939699 15971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:41.939853 15971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:41.940047 15971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:09:41.940325 15971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:09:41.940376 15971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:41.940413 15971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:09:41.940440 15971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:41.946157 15971 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:41463
I20260925 13:09:41.946195 16099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:41463 every 8 connection(s)
I20260925 13:09:41.946522 15971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:09:41.947497 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:41.947151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:51601)
0925 13:09:41.947293 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:41.947370 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
W20260925 13:09:41.947891 16100 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:44859 (0 consecutive failures): Network error: Failed to ping master at 127.0.206.126:44859: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
I20260925 13:09:41.948123 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:41.948056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:45827)
0925 13:09:41.948083 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:41.948107 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260925 13:09:41.948383 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:41.948311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:35939)
0925 13:09:41.948350 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:41.948370 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 13:09:41.948494 16100 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 13:09:41.950836 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 15971
I20260925 13:09:42.797920 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:42.797716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:52151)
0925 13:09:42.797831 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:42.797864 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":52,"threads_started":1}
I20260925 13:09:42.837810 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:42.837622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:46573)
0925 13:09:42.837740 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:42.837768 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260925 13:09:42.948959 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:42.948755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:34327)
0925 13:09:42.948882 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:42.948914 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":54,"threads_started":1}
I20260925 13:09:43.114291 16107 raft_consensus.cc:482] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:09:43.114455 16107 raft_consensus.cc:504] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } } peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } }
I20260925 13:09:43.114890 16107 leader_election.cc:288] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66:34897), 034c6426204c4846a2ea10f6233bca00 (127.0.206.67:41463)
Sep 25 13:09:43 dist-test-slave-chkf krb5kdc[14535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.66@KRBTEST.COM
Sep 25 13:09:43 dist-test-slave-chkf krb5kdc[14535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.67@KRBTEST.COM
I20260925 13:09:43.125124 16110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:43.115190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:44625 (local address 127.0.206.67:41463)
0925 13:09:43.115323 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:43.115334 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:43.115371 (+ 37us) server_negotiation.cc:407] Connection header received
0925 13:09:43.115784 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:43.115787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:43.115822 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:43.115971 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:43.116850 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:43.117836 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:43.118795 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:43.119183 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:43.119294 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:43.121898 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:43.121902 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:43.121902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:43.121912 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:43.123625 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:43.124003 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:43.124005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:43.124006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:43.124120 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:43.124320 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:43.124321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:43.124322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:43.124531 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:43.124635 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:43.124896 (+ 261us) server_negotiation.cc:299] Negotiation successful
0925 13:09:43.125017 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":58,"threads_started":1}
I20260925 13:09:43.125124 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:43.115008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:34897 (local address 127.0.206.65:44511)
0925 13:09:43.115064 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:43.115075 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:43.115127 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:43.115979 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:43.115981 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:43.116543 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:43.116724 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:43.116730 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:43.117947 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:43.117948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:43.118681 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:43.118684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:43.119281 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:43.119282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:43.119377 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:43.119931 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:43.119939 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:43.121417 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:43.123499 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:43.123506 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:43.123508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:43.123832 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:43.124089 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:43.124090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:43.124090 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:43.124225 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:43.124598 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:43.124604 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:43.124734 (+ 130us) client_negotiation.cc:769] Sending connection context
0925 13:09:43.124831 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 13:09:43.125010 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":53}
I20260925 13:09:43.125145 16109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:43.115112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:44511 (local address 127.0.206.66:34897)
0925 13:09:43.115275 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:43.115287 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:43.115322 (+ 35us) server_negotiation.cc:407] Connection header received
0925 13:09:43.115758 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:43.115759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:43.115799 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:43.115934 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:43.116849 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:43.117837 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:43.118796 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:43.119186 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:43.119305 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:43.121539 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:43.121543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:43.121543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:43.121553 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:43.123413 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:43.123907 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:43.123910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:43.123911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:43.124017 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:43.124297 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:43.124298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:43.124299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:43.124526 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:43.124617 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:43.124888 (+ 271us) server_negotiation.cc:299] Negotiation successful
0925 13:09:43.125006 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":64,"threads_started":1}
I20260925 13:09:43.125869 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:43.115008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:41463 (local address 127.0.206.65:44625)
0925 13:09:43.115205 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:43.115214 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:43.115264 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:43.115983 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:43.115984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:43.116546 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:43.116724 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:43.116730 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:43.117922 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:43.117923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:43.118681 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:43.118683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:43.119269 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:43.119270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:43.119376 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:43.119983 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:43.119988 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:43.121793 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:43.123716 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:43.123716 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:43.123717 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:43.123927 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:43.124189 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:43.124190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:43.124190 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:43.124241 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:43.124622 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:43.124623 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:43.124729 (+ 106us) client_negotiation.cc:769] Sending connection context
0925 13:09:43.124831 (+ 102us) client_negotiation.cc:240] Negotiation successful
0925 13:09:43.125088 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":145,"thread_start_us":69,"threads_started":1}
W20260925 13:09:43.125998 15723 leader_election.cc:341] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66:34897): Not found: Tablet not found: b15bf61c24bb4c679d1a7f19c99f3273
W20260925 13:09:43.126519 15721 leader_election.cc:341] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 034c6426204c4846a2ea10f6233bca00 (127.0.206.67:41463): Not found: Tablet not found: b15bf61c24bb4c679d1a7f19c99f3273
I20260925 13:09:43.126564 15721 leader_election.cc:302] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 03a1a95e4d5b4a89b0d3d80a5c9743e4; no voters: 034c6426204c4846a2ea10f6233bca00, 6a9f8a96e5aa4c89b0a6242e211f8dfc
I20260925 13:09:43.126667 16107 raft_consensus.cc:2726] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:09:43.798555 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:43.798338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:55489)
0925 13:09:43.798472 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:43.798506 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":62,"threads_started":1}
I20260925 13:09:43.838428 16112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:43.838232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:55937)
0925 13:09:43.838357 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:43.838390 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":56,"threads_started":1}
I20260925 13:09:43.949627 16113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:43.949369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:48633)
0925 13:09:43.949556 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:43.949593 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":67,"threads_started":1}
I20260925 13:09:44.789693 16114 raft_consensus.cc:482] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:09:44.789788 16114 raft_consensus.cc:504] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } } peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } }
I20260925 13:09:44.789996 16114 leader_election.cc:288] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66:34897), 034c6426204c4846a2ea10f6233bca00 (127.0.206.67:41463)
W20260925 13:09:44.790427 15723 leader_election.cc:341] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66:34897): Not found: Tablet not found: b15bf61c24bb4c679d1a7f19c99f3273
W20260925 13:09:44.790490 15721 leader_election.cc:341] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 034c6426204c4846a2ea10f6233bca00 (127.0.206.67:41463): Not found: Tablet not found: b15bf61c24bb4c679d1a7f19c99f3273
I20260925 13:09:44.790518 15723 leader_election.cc:302] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 03a1a95e4d5b4a89b0d3d80a5c9743e4; no voters: 034c6426204c4846a2ea10f6233bca00, 6a9f8a96e5aa4c89b0a6242e211f8dfc
I20260925 13:09:44.790623 16114 raft_consensus.cc:2726] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:09:44.799161 16115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:44.798983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:50167)
0925 13:09:44.799087 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:44.799122 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":67,"threads_started":1}
I20260925 13:09:44.839087 16116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:44.838913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:53897)
0925 13:09:44.839017 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:44.839050 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":53,"threads_started":1}
I20260925 13:09:44.950336 16117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:44.950142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:44843)
0925 13:09:44.950268 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:44.950302 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260925 13:09:45.799882 16118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:45.799666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:50161)
0925 13:09:45.799802 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:45.799836 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
I20260925 13:09:45.839673 16119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:45.839481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:57319)
0925 13:09:45.839604 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:45.839636 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":57,"threads_started":1}
I20260925 13:09:45.951063 16120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:45.950880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:36363)
0925 13:09:45.950999 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:45.951030 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260925 13:09:46.800690 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:46.800468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:54107)
0925 13:09:46.800612 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:46.800647 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
I20260925 13:09:46.840339 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:46.840136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:59647)
0925 13:09:46.840275 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:46.840306 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260925 13:09:46.951645 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:46.951417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:44705)
0925 13:09:46.951577 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:46.951610 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":81,"threads_started":1}
I20260925 13:09:47.262828 16124 raft_consensus.cc:482] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:09:47.262956 16124 raft_consensus.cc:504] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } } peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } }
I20260925 13:09:47.263168 16124 leader_election.cc:288] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66:34897), 034c6426204c4846a2ea10f6233bca00 (127.0.206.67:41463)
W20260925 13:09:47.263514 15723 leader_election.cc:341] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66:34897): Not found: Tablet not found: b15bf61c24bb4c679d1a7f19c99f3273
W20260925 13:09:47.263590 15721 leader_election.cc:341] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 034c6426204c4846a2ea10f6233bca00 (127.0.206.67:41463): Not found: Tablet not found: b15bf61c24bb4c679d1a7f19c99f3273
I20260925 13:09:47.263650 15721 leader_election.cc:302] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 03a1a95e4d5b4a89b0d3d80a5c9743e4; no voters: 034c6426204c4846a2ea10f6233bca00, 6a9f8a96e5aa4c89b0a6242e211f8dfc
I20260925 13:09:47.263741 16124 raft_consensus.cc:2726] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:09:47.801409 16125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:47.801156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:45739)
0925 13:09:47.801308 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:47.801344 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260925 13:09:47.840939 16126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:47.840723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:55407)
0925 13:09:47.840868 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:47.840901 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":58,"threads_started":1}
I20260925 13:09:47.952207 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:47.952029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:57631)
0925 13:09:47.952140 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:47.952173 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260925 13:09:48.802035 16128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:48.801817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:41573)
0925 13:09:48.801953 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:48.801989 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260925 13:09:48.841665 16129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:48.841447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:59827)
0925 13:09:48.841592 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:48.841625 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":65,"threads_started":1}
I20260925 13:09:48.952777 16130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:48.952600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:51863)
0925 13:09:48.952712 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:48.952744 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260925 13:09:49.354769 16131 raft_consensus.cc:482] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:09:49.354862 16131 raft_consensus.cc:504] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } } peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } }
I20260925 13:09:49.355031 16131 leader_election.cc:288] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66:34897), 034c6426204c4846a2ea10f6233bca00 (127.0.206.67:41463)
W20260925 13:09:49.355413 15723 leader_election.cc:341] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66:34897): Not found: Tablet not found: b15bf61c24bb4c679d1a7f19c99f3273
W20260925 13:09:49.355489 15721 leader_election.cc:341] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 034c6426204c4846a2ea10f6233bca00 (127.0.206.67:41463): Not found: Tablet not found: b15bf61c24bb4c679d1a7f19c99f3273
I20260925 13:09:49.355551 15723 leader_election.cc:302] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 03a1a95e4d5b4a89b0d3d80a5c9743e4; no voters: 034c6426204c4846a2ea10f6233bca00, 6a9f8a96e5aa4c89b0a6242e211f8dfc
I20260925 13:09:49.355630 16131 raft_consensus.cc:2726] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:09:49.802968 16132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:49.802690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:39249)
0925 13:09:49.802867 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:49.802918 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":82,"threads_started":1}
I20260925 13:09:49.842329 16133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:49.842108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:34607)
0925 13:09:49.842244 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:49.842282 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260925 13:09:49.953464 16134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:49.953245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:55103)
0925 13:09:49.953382 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:49.953426 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":58,"threads_started":1}
I20260925 13:09:50.803683 16135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:50.803426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:33547)
0925 13:09:50.803585 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:50.803627 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":58,"threads_started":1}
I20260925 13:09:50.843122 16136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:50.842934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:51711)
0925 13:09:50.843050 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:50.843083 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260925 13:09:50.954090 16137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:50.953897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:55921)
0925 13:09:50.954013 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:50.954054 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260925 13:09:51.804317 16138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:51.804115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:59529)
0925 13:09:51.804235 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:51.804274 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260925 13:09:51.843703 16139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:51.843516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:45643)
0925 13:09:51.843633 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:51.843667 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
I20260925 13:09:51.954684 16140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:51.954486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:38989)
0925 13:09:51.954612 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:51.954647 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":55,"threads_started":1}
I20260925 13:09:52.804975 16141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:52.804778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:37155)
0925 13:09:52.804881 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:52.804915 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":50,"threads_started":1}
I20260925 13:09:52.844236 16142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:52.844060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:48093)
0925 13:09:52.844168 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:52.844200 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":49,"threads_started":1}
I20260925 13:09:52.955329 16143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:52.955142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:34239)
0925 13:09:52.955265 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:52.955295 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":68,"threads_started":1}
I20260925 13:09:53.805615 16144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:53.805384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:50415)
0925 13:09:53.805518 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:53.805569 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260925 13:09:53.844900 16145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:53.844705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:34431)
0925 13:09:53.844833 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:53.844862 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260925 13:09:53.914081 16146 raft_consensus.cc:482] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:09:53.914170 16146 raft_consensus.cc:504] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } } peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } }
I20260925 13:09:53.914379 16146 leader_election.cc:288] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66:34897), 034c6426204c4846a2ea10f6233bca00 (127.0.206.67:41463)
W20260925 13:09:53.914767 15721 leader_election.cc:341] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 034c6426204c4846a2ea10f6233bca00 (127.0.206.67:41463): Not found: Tablet not found: b15bf61c24bb4c679d1a7f19c99f3273
W20260925 13:09:53.914844 15723 leader_election.cc:341] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66:34897): Not found: Tablet not found: b15bf61c24bb4c679d1a7f19c99f3273
I20260925 13:09:53.914881 15723 leader_election.cc:302] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 03a1a95e4d5b4a89b0d3d80a5c9743e4; no voters: 034c6426204c4846a2ea10f6233bca00, 6a9f8a96e5aa4c89b0a6242e211f8dfc
I20260925 13:09:53.914959 16146 raft_consensus.cc:2726] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:09:53.955922 16147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:53.955722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:53615)
0925 13:09:53.955845 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:53.955876 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260925 13:09:54.806269 16148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:54.806072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:35139)
0925 13:09:54.806195 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:54.806227 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
I20260925 13:09:54.845599 16149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:54.845309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:54815)
0925 13:09:54.845525 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:54.845560 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":69,"threads_started":1}
I20260925 13:09:54.956600 16150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:54.956305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:53049)
0925 13:09:54.956532 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:54.956566 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":76,"threads_started":1}
I20260925 13:09:55.806938 16151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:55.806686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:49295)
0925 13:09:55.806857 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:55.806892 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":103,"threads_started":1}
I20260925 13:09:55.846264 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:55.846065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:54081)
0925 13:09:55.846195 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:55.846224 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260925 13:09:55.957222 16153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:55.957027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:57901)
0925 13:09:55.957152 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:55.957183 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260925 13:09:56.758014 16154 raft_consensus.cc:482] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:09:56.758116 16154 raft_consensus.cc:504] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9f8a96e5aa4c89b0a6242e211f8dfc" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34897 } } peers { permanent_uuid: "034c6426204c4846a2ea10f6233bca00" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41463 } } peers { permanent_uuid: "03a1a95e4d5b4a89b0d3d80a5c9743e4" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41397 } }
I20260925 13:09:56.758329 16154 leader_election.cc:288] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66:34897), 034c6426204c4846a2ea10f6233bca00 (127.0.206.67:41463)
W20260925 13:09:56.758744 15723 leader_election.cc:341] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6a9f8a96e5aa4c89b0a6242e211f8dfc (127.0.206.66:34897): Not found: Tablet not found: b15bf61c24bb4c679d1a7f19c99f3273
W20260925 13:09:56.758795 15721 leader_election.cc:341] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 034c6426204c4846a2ea10f6233bca00 (127.0.206.67:41463): Not found: Tablet not found: b15bf61c24bb4c679d1a7f19c99f3273
I20260925 13:09:56.758822 15723 leader_election.cc:302] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 03a1a95e4d5b4a89b0d3d80a5c9743e4; no voters: 034c6426204c4846a2ea10f6233bca00, 6a9f8a96e5aa4c89b0a6242e211f8dfc
I20260925 13:09:56.758896 16154 raft_consensus.cc:2726] T b15bf61c24bb4c679d1a7f19c99f3273 P 03a1a95e4d5b4a89b0d3d80a5c9743e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:09:56.807669 16155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:56.807457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.65:51203)
0925 13:09:56.807585 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:56.807617 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260925 13:09:56.847028 16156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:56.846821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.66:40679)
0925 13:09:56.846950 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:56.846985 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260925 13:09:56.955586 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 15675
I20260925 13:09:56.957864 16157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:56.957687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44859 (local address 127.0.206.67:37169)
0925 13:09:56.957794 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:56.957827 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:44859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":66,"threads_started":1}
I20260925 13:09:56.960644 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 15838
I20260925 13:09:56.964962 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 15971
2026-09-25T13:09:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18644 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:09:57 dist-test-slave-chkf krb5kdc[16160](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:09:57 dist-test-slave-chkf krb5kdc[16160](info): set up 2 sockets
Sep 25 13:09:57 dist-test-slave-chkf krb5kdc[16160](info): commencing operation
krb5kdc: starting...
W20260925 13:09:58.002532 825 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:09:58 dist-test-slave-chkf krb5kdc[16160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:09:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:09:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:09:58.114909 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36617
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:36617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:58.192190 16173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:58.192420 16173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:58.192456 16173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:58.193593 16173 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 13:09:58.193670 16173 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 13:09:58.193689 16173 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 13:09:58.193707 16173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:09:58.193742 16173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:58.193773 16173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:58.193796 16173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:09:58.193828 16173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:09:58.195255 16173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:36617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:58.195505 16173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:58.195710 16173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:58.200830 16181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:58.200847 16178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:58.200949 16179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:58.200963 16173 server_base.cc:1034] running on GCE node
I20260925 13:09:58.201328 16173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:58.201596 16173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:58.202762 16173 hybrid_clock.cc:648] HybridClock initialized: now 1790341798202753 us; error 29 us; skew 500 ppm
Sep 25 13:09:58 dist-test-slave-chkf krb5kdc[16160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341798, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:58.205042 16173 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:09:58.205488 16173 webserver.cc:460] Webserver started at http://127.0.206.126:37719/ using document root <none> and password file <none>
I20260925 13:09:58.205703 16173 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:58.205768 16173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:58.205917 16173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:09:58.206983 16173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "98bf9d7fe1c044869a6a8394ab50df41"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "6dc2068df3a58354e5ddb0de00ab9764"
server_key_iv: "914680682f363c53b392182838596764"
server_key_version: "encryptionkey@0"
I20260925 13:09:58.207335 16173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "98bf9d7fe1c044869a6a8394ab50df41"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "6dc2068df3a58354e5ddb0de00ab9764"
server_key_iv: "914680682f363c53b392182838596764"
server_key_version: "encryptionkey@0"
I20260925 13:09:58.208655 16173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:09:58.209352 16188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:58.209545 16173 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:09:58.209642 16173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "98bf9d7fe1c044869a6a8394ab50df41"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "6dc2068df3a58354e5ddb0de00ab9764"
server_key_iv: "914680682f363c53b392182838596764"
server_key_version: "encryptionkey@0"
I20260925 13:09:58.209719 16173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:58.216614 16173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:58.218254 16173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:58.218400 16173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:58.222244 16173 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:36617
I20260925 13:09:58.222273 16240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:36617 every 8 connection(s)
I20260925 13:09:58.222587 16173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:09:58.223385 16241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:58.225430 16241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: Bootstrap starting.
I20260925 13:09:58.225944 16241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:58.226178 16241 log.cc:824] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:58.226831 16241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: No bootstrap required, opened a new log
I20260925 13:09:58.228295 16241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bf9d7fe1c044869a6a8394ab50df41" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36617 } }
I20260925 13:09:58.228420 16241 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:58.228442 16241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98bf9d7fe1c044869a6a8394ab50df41, State: Initialized, Role: FOLLOWER
I20260925 13:09:58.228576 16241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bf9d7fe1c044869a6a8394ab50df41" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36617 } }
I20260925 13:09:58.228680 16241 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:09:58.228716 16241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:09:58.228751 16241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:58.228981 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 16173
I20260925 13:09:58.229103 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:09:58.229277 16241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bf9d7fe1c044869a6a8394ab50df41" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36617 } }
I20260925 13:09:58.229306 825 external_mini_cluster.cc:1428] Setting key 47e82ca7d98fa97ecff79af42a81bd4e
I20260925 13:09:58.229383 16241 leader_election.cc:302] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98bf9d7fe1c044869a6a8394ab50df41; no voters:
I20260925 13:09:58.229540 16241 leader_election.cc:288] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:09:58.229568 16244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:09:58.229693 16244 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 1 LEADER]: Becoming Leader. State: Replica: 98bf9d7fe1c044869a6a8394ab50df41, State: Running, Role: LEADER
I20260925 13:09:58.229810 16241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:09:58.229842 16244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bf9d7fe1c044869a6a8394ab50df41" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36617 } }
I20260925 13:09:58.230441 16244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98bf9d7fe1c044869a6a8394ab50df41. Latest consensus state: current_term: 1 leader_uuid: "98bf9d7fe1c044869a6a8394ab50df41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bf9d7fe1c044869a6a8394ab50df41" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36617 } } }
I20260925 13:09:58.230448 16246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98bf9d7fe1c044869a6a8394ab50df41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bf9d7fe1c044869a6a8394ab50df41" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36617 } } }
I20260925 13:09:58.230732 16244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [sys.catalog]: This master's current role is: LEADER
I20260925 13:09:58.231112 16246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [sys.catalog]: This master's current role is: LEADER
W20260925 13:09:58.231737 16262 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:09:58.232049 16262 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:09:58.232120 16257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:09:58.232268 16257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:09:58.233716 16257 catalog_manager.cc:1347] Generated new cluster ID: ff4631fa05b345f794bd3027117e9280
I20260925 13:09:58.233769 16257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 13:09:58 dist-test-slave-chkf krb5kdc[16160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:58.240614 16248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:58.229953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48396 (local address 127.0.206.126:36617)
0925 13:09:58.230134 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:58.230149 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:58.230943 (+ 794us) server_negotiation.cc:407] Connection header received
0925 13:09:58.231166 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:58.231174 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:58.231233 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:58.231375 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:58.231981 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.233127 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.234069 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.234490 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.234616 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:58.236886 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:58.236890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:58.236890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:58.236901 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:58.239097 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:58.239499 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:58.239502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:58.239503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:58.239585 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:58.239803 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:58.239804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:58.239804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:58.240031 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:58.240188 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:58.240292 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 13:09:58.240428 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260925 13:09:58.277535 16257 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:09:58.278079 16257 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:09:58.285678 16257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: Generated new TSK 0
I20260925 13:09:58.285871 16257 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260925 13:09:58.313232 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36617
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:58.390995 16267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:58.391201 16267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:58.391237 16267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:58.392344 16267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:58.392408 16267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:58.392452 16267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:09:58.393837 16267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:58.394073 16267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:58.394284 16267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:58.399264 16272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:58.399405 16273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:58.399360 16267 server_base.cc:1034] running on GCE node
W20260925 13:09:58.399515 16275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:58.399741 16267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:58.400000 16267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:58.401166 16267 hybrid_clock.cc:648] HybridClock initialized: now 1790341798401162 us; error 35 us; skew 500 ppm
Sep 25 13:09:58 dist-test-slave-chkf krb5kdc[16160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341798, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:58.403679 16267 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:09:58.404036 16267 webserver.cc:460] Webserver started at http://127.0.206.65:33403/ using document root <none> and password file <none>
I20260925 13:09:58.404210 16267 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:58.404249 16267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:58.404335 16267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:09:58.405262 16267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "2587f73a3c82462b8821e825761ef628"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "caa64392ce8701b60b5978a5cfd6569a"
server_key_iv: "498618e0fb6671293c960d3ff1f8be1b"
server_key_version: "encryptionkey@0"
I20260925 13:09:58.405552 16267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "2587f73a3c82462b8821e825761ef628"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "caa64392ce8701b60b5978a5cfd6569a"
server_key_iv: "498618e0fb6671293c960d3ff1f8be1b"
server_key_version: "encryptionkey@0"
I20260925 13:09:58.406790 16267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:09:58.407452 16282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:58.407635 16267 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:09:58.407706 16267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "2587f73a3c82462b8821e825761ef628"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "caa64392ce8701b60b5978a5cfd6569a"
server_key_iv: "498618e0fb6671293c960d3ff1f8be1b"
server_key_version: "encryptionkey@0"
I20260925 13:09:58.407793 16267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:58.433358 16267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:58.435731 16267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:58.435899 16267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:58.436173 16267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:09:58.436543 16267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:09:58.436578 16267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:58.436600 16267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:09:58.436643 16267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:58.442332 16267 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:40025
I20260925 13:09:58.442380 16395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:40025 every 8 connection(s)
I20260925 13:09:58.442677 16267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:09:58.447695 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 16267
I20260925 13:09:58.447810 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:09:58.448066 825 external_mini_cluster.cc:1428] Setting key e08c69b8e4ad2b9c2173528fe5fc7cb0
Sep 25 13:09:58 dist-test-slave-chkf krb5kdc[16160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341798, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:58.451973 16248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:58.443368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:43865 (local address 127.0.206.126:36617)
0925 13:09:58.443409 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:58.443411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:58.443618 (+ 207us) server_negotiation.cc:407] Connection header received
0925 13:09:58.444087 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:58.444088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:58.444099 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:58.444157 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:58.445004 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.445509 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.446475 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.446683 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.446762 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:58.449115 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:58.449116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:58.449117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:58.449125 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:58.450614 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:58.451035 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:58.451036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:58.451037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:58.451098 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:58.451347 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:58.451348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:58.451349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:58.451461 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:58.451531 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:58.451800 (+ 269us) server_negotiation.cc:299] Negotiation successful
0925 13:09:58.451885 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 13:09:58.452162 16398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:58.443338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:43865)
0925 13:09:58.443498 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:58.443538 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:58.444001 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:58.444208 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:58.444211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:58.444739 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:58.444916 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:58.444920 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.445600 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.445600 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:58.446380 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:58.446382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.446793 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.446794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:58.446903 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:58.447311 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:58.447319 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:58.449027 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:58.450723 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:58.450730 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:58.450732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:58.450943 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:58.451169 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:58.451170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:58.451171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:58.451279 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:58.451533 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:58.451540 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:58.451672 (+ 132us) client_negotiation.cc:769] Sending connection context
0925 13:09:58.451779 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 13:09:58.451922 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260925 13:09:58.452798 16396 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36617
I20260925 13:09:58.452872 16396 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:58.453055 16396 heartbeater.cc:511] Master 127.0.206.126:36617 requested a full tablet report, sending...
I20260925 13:09:58.453591 16205 ts_manager.cc:194] Registered new tserver with Master: 2587f73a3c82462b8821e825761ef628 (127.0.206.65:40025)
I20260925 13:09:58.454190 16205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:43865
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260925 13:09:58.472990 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36617
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:58.550129 16401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:58.550307 16401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:58.550328 16401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:58.551451 16401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:58.551491 16401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:58.551541 16401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:09:58.552932 16401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:58.553150 16401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:58.553357 16401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:58.558183 16407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:58.558218 16406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:58.558293 16401 server_base.cc:1034] running on GCE node
W20260925 13:09:58.558192 16409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:58.558627 16401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:58.558881 16401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:58.560014 16401 hybrid_clock.cc:648] HybridClock initialized: now 1790341798560003 us; error 28 us; skew 500 ppm
Sep 25 13:09:58 dist-test-slave-chkf krb5kdc[16160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341798, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:58.562294 16401 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:09:58.562686 16401 webserver.cc:460] Webserver started at http://127.0.206.66:34287/ using document root <none> and password file <none>
I20260925 13:09:58.562935 16401 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:58.563001 16401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:58.563170 16401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:09:58.564116 16401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "7099cb9663264a61bacd4a1c83989c26"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "6946de8ff053c97ddf4535605427b6c7"
server_key_iv: "8cecc3d4ba7b9583a8754a51c4cf7859"
server_key_version: "encryptionkey@0"
I20260925 13:09:58.564463 16401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "7099cb9663264a61bacd4a1c83989c26"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "6946de8ff053c97ddf4535605427b6c7"
server_key_iv: "8cecc3d4ba7b9583a8754a51c4cf7859"
server_key_version: "encryptionkey@0"
I20260925 13:09:58.565701 16401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:09:58.566342 16416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:58.566499 16401 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:09:58.566593 16401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "7099cb9663264a61bacd4a1c83989c26"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "6946de8ff053c97ddf4535605427b6c7"
server_key_iv: "8cecc3d4ba7b9583a8754a51c4cf7859"
server_key_version: "encryptionkey@0"
I20260925 13:09:58.566677 16401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:58.600090 16401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:58.601811 16401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:58.601965 16401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:58.602176 16401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:09:58.602459 16401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:09:58.602491 16401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:58.602532 16401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:09:58.602561 16401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:58.608132 16401 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:38717
I20260925 13:09:58.608201 16529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:38717 every 8 connection(s)
I20260925 13:09:58.608502 16401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
Sep 25 13:09:58 dist-test-slave-chkf krb5kdc[16160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341798, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:58.616835 16248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:58.609213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:45029 (local address 127.0.206.126:36617)
0925 13:09:58.609255 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:58.609256 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:58.609450 (+ 194us) server_negotiation.cc:407] Connection header received
0925 13:09:58.609944 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:58.609946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:58.609958 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:58.610013 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:58.611042 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.611590 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.612507 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.612691 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.612745 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:58.614604 (+ 1859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:58.614605 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:58.614606 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:58.614611 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:58.615799 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:58.616131 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:58.616132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:58.616132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:58.616177 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:58.616368 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:58.616368 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:58.616368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:58.616455 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:58.616501 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:58.616716 (+ 215us) server_negotiation.cc:299] Negotiation successful
0925 13:09:58.616776 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:09:58.616945 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:58.609175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:45029)
0925 13:09:58.609335 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:58.609374 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:58.609832 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:58.610073 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:58.610077 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:58.610690 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:58.610934 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:58.610939 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.611674 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.611675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:58.612438 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:58.612440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.612760 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.612761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:58.612852 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:58.613249 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:58.613256 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:58.614525 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:58.615876 (+ 1351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:58.615881 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:58.615882 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:58.616071 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:58.616229 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:58.616229 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:58.616230 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:58.616316 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:58.616504 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:58.616507 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:58.616600 (+ 93us) client_negotiation.cc:769] Sending connection context
0925 13:09:58.616681 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 13:09:58.616780 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260925 13:09:58.617360 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 16401
I20260925 13:09:58.617440 16530 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36617
I20260925 13:09:58.617502 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:09:58.617532 16530 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:58.617700 16530 heartbeater.cc:511] Master 127.0.206.126:36617 requested a full tablet report, sending...
I20260925 13:09:58.617717 825 external_mini_cluster.cc:1428] Setting key 436cf4a5da79e357f56f1f4a7e0d9ced
I20260925 13:09:58.618120 16205 ts_manager.cc:194] Registered new tserver with Master: 7099cb9663264a61bacd4a1c83989c26 (127.0.206.66:38717)
I20260925 13:09:58.618609 16205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:45029
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260925 13:09:58.641386 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36617
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:09:58.719873 16535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:09:58.720048 16535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:09:58.720070 16535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:09:58.721107 16535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:09:58.721138 16535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:09:58.721163 16535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:09:58.722458 16535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:09:58.722636 16535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:09:58.722856 16535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:09:58.727432 16543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:58.727443 16541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:09:58.727505 16540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:09:58.727725 16535 server_base.cc:1034] running on GCE node
I20260925 13:09:58.727911 16535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:09:58.728116 16535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:09:58.729260 16535 hybrid_clock.cc:648] HybridClock initialized: now 1790341798729251 us; error 25 us; skew 500 ppm
Sep 25 13:09:58 dist-test-slave-chkf krb5kdc[16160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341798, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:09:58.731874 16535 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:09:58.732291 16535 webserver.cc:460] Webserver started at http://127.0.206.67:45995/ using document root <none> and password file <none>
I20260925 13:09:58.732511 16535 fs_manager.cc:359] Metadata directory not provided
I20260925 13:09:58.732553 16535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:09:58.732697 16535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:09:58.733659 16535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "c7418852b93f40139f3b93e08840d0df"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "4c65fa7b538a7bddb868277065beb27d"
server_key_iv: "9bdb79ffcbabff4b94445dfaf3cb9c59"
server_key_version: "encryptionkey@0"
I20260925 13:09:58.733999 16535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "c7418852b93f40139f3b93e08840d0df"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "4c65fa7b538a7bddb868277065beb27d"
server_key_iv: "9bdb79ffcbabff4b94445dfaf3cb9c59"
server_key_version: "encryptionkey@0"
I20260925 13:09:58.735391 16535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:09:58.736078 16550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:58.736274 16535 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:09:58.736358 16535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "c7418852b93f40139f3b93e08840d0df"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "4c65fa7b538a7bddb868277065beb27d"
server_key_iv: "9bdb79ffcbabff4b94445dfaf3cb9c59"
server_key_version: "encryptionkey@0"
I20260925 13:09:58.736424 16535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:09:58.749380 16535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:09:58.751223 16535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:09:58.751375 16535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:09:58.751590 16535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:09:58.751880 16535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:09:58.751933 16535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:58.751965 16535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:09:58.751993 16535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:09:58.757843 16535 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:43321
I20260925 13:09:58.757884 16663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:43321 every 8 connection(s)
I20260925 13:09:58.758175 16535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
Sep 25 13:09:58 dist-test-slave-chkf krb5kdc[16160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341798, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:09:58.765403 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 16535
I20260925 13:09:58.765498 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:09:58.765713 825 external_mini_cluster.cc:1428] Setting key 664fd05179a051f792420d5a4f949857
I20260925 13:09:58.766642 16248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:58.758873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:40085 (local address 127.0.206.126:36617)
0925 13:09:58.758913 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:58.758915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:58.759091 (+ 176us) server_negotiation.cc:407] Connection header received
0925 13:09:58.759598 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:58.759598 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:58.759608 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:58.759662 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:58.760485 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.760961 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.761872 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.762043 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.762100 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:58.764117 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:58.764118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:58.764119 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:58.764125 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:58.765248 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:58.765707 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:58.765708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:58.765708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:58.765774 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:58.766021 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:58.766022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:58.766022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:58.766140 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:58.766217 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:58.766471 (+ 254us) server_negotiation.cc:299] Negotiation successful
0925 13:09:58.766546 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:09:58.766834 16666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:58.758849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:40085)
0925 13:09:58.758989 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:58.759026 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:58.759509 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:58.759713 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:58.759716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:58.760241 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:09:58.760416 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:58.760420 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.761043 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.761044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:58.761804 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:58.761805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.762119 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.762119 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:58.762223 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:58.762656 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:09:58.762664 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:09:58.764027 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:09:58.765323 (+ 1296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:58.765330 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:58.765331 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:58.765630 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:58.765858 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:09:58.765858 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:09:58.765859 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:09:58.765951 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:09:58.766209 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:09:58.766214 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:09:58.766343 (+ 129us) client_negotiation.cc:769] Sending connection context
0925 13:09:58.766448 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 13:09:58.766591 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":80,"threads_started":1}
I20260925 13:09:58.767217 16664 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36617
I20260925 13:09:58.767300 16664 heartbeater.cc:464] Registering TS with master...
I20260925 13:09:58.767427 16664 heartbeater.cc:511] Master 127.0.206.126:36617 requested a full tablet report, sending...
I20260925 13:09:58.767756 16205 ts_manager.cc:194] Registered new tserver with Master: c7418852b93f40139f3b93e08840d0df (127.0.206.67:43321)
I20260925 13:09:58.768123 16205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:40085
I20260925 13:09:58.777253 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:09:58.783668 16248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:58.778499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48402 (local address 127.0.206.126:36617)
0925 13:09:58.778568 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:58.778572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:58.778627 (+ 55us) server_negotiation.cc:407] Connection header received
0925 13:09:58.778666 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:58.778667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:58.778674 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:58.778778 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:09:58.779264 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.779698 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.780351 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.780513 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.780557 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:58.781654 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:09:58.781654 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:09:58.781654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:09:58.781659 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:09:58.782808 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:58.783101 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:58.783102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:58.783102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:58.783150 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:09:58.783277 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:09:58.783277 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:09:58.783277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:09:58.783396 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:09:58.783469 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:58.783535 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 13:09:58.783586 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 13:09:58.785010 16205 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:09:58.786785 16205 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:09:58.793004 16676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:58.788789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:43321 (local address 127.0.0.1:60646)
0925 13:09:58.788925 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:58.788937 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:58.788999 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:58.789517 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:58.789518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:58.789520 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:58.789576 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:58.789578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.790513 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.790514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:58.791822 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:58.791824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.792716 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.792718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:58.792797 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:58.792811 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 13:09:58.792857 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 13:09:58.792905 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":48,"thread_start_us":82,"threads_started":1}
I20260925 13:09:58.793255 16675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:58.788757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38717 (local address 127.0.0.1:32926)
0925 13:09:58.788900 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:58.788937 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:58.788999 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:58.789337 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:58.789340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:58.789342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:58.789402 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:58.789404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.790722 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.790723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:58.791524 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:58.791527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.793006 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.793007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:58.793094 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:58.793103 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 13:09:58.793133 (+ 30us) client_negotiation.cc:240] Negotiation successful
0925 13:09:58.793185 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":4,"thread_start_us":49,"threads_started":1}
I20260925 13:09:58.793419 16679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:58.789017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34836 (local address 127.0.206.65:40025)
0925 13:09:58.789395 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:58.789399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:58.789408 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:09:58.789442 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:58.789444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:58.789457 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:58.789532 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:58.789660 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.790723 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.791968 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.792685 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.792883 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:58.793000 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:58.793299 (+ 299us) server_negotiation.cc:299] Negotiation successful
0925 13:09:58.793339 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":82,"threads_started":1}
I20260925 13:09:58.793592 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:58.789188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60646 (local address 127.0.206.67:43321)
0925 13:09:58.789334 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:58.789339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:58.789350 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:09:58.789386 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:58.789388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:58.789404 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:58.789662 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:58.789723 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.790414 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.791903 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.792618 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.793375 (+ 757us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:58.793437 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:58.793464 (+ 27us) server_negotiation.cc:299] Negotiation successful
0925 13:09:58.793509 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260925 13:09:58.793670 16677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:58.788789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:40025 (local address 127.0.0.1:34836)
0925 13:09:58.789097 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:58.789106 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:58.789143 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:58.789527 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:58.789528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:58.789530 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:58.789568 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:58.789570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.790805 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.790806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:58.791896 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:58.791897 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.793035 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.793036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:58.793094 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:58.793105 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 13:09:58.793189 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 13:09:58.793238 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":4,"thread_start_us":89,"threads_started":1}
I20260925 13:09:58.793746 16678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:58.789021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32926 (local address 127.0.206.66:38717)
0925 13:09:58.789177 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:58.789182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:58.789191 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:09:58.789227 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:58.789229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:58.789249 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:58.789335 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:58.789469 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.790588 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.791626 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:58.792923 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:58.793543 (+ 620us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:58.793600 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:58.793625 (+ 25us) server_negotiation.cc:299] Negotiation successful
0925 13:09:58.793668 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260925 13:09:58.794632 16598 tablet_service.cc:1467] Processing CreateTablet for tablet a9636e736394410196a076b097aa3aff (DEFAULT_TABLE table=test-table [id=8a1102302c864961a51675867a13ea63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:09:58.795014 16598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9636e736394410196a076b097aa3aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:58.795367 16464 tablet_service.cc:1467] Processing CreateTablet for tablet a9636e736394410196a076b097aa3aff (DEFAULT_TABLE table=test-table [id=8a1102302c864961a51675867a13ea63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:09:58.795666 16464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9636e736394410196a076b097aa3aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:58.798197 16330 tablet_service.cc:1467] Processing CreateTablet for tablet a9636e736394410196a076b097aa3aff (DEFAULT_TABLE table=test-table [id=8a1102302c864961a51675867a13ea63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:09:58.798656 16330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9636e736394410196a076b097aa3aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:09:58.799105 16682 tablet_bootstrap.cc:492] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26: Bootstrap starting.
I20260925 13:09:58.799849 16682 tablet_bootstrap.cc:654] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:58.800544 16682 log.cc:824] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:58.801385 16682 tablet_bootstrap.cc:492] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26: No bootstrap required, opened a new log
I20260925 13:09:58.801476 16682 ts_tablet_manager.cc:1397] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:09:58.801816 16681 tablet_bootstrap.cc:492] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df: Bootstrap starting.
I20260925 13:09:58.802332 16684 tablet_bootstrap.cc:492] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628: Bootstrap starting.
I20260925 13:09:58.802466 16681 tablet_bootstrap.cc:654] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:58.803117 16684 tablet_bootstrap.cc:654] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628: Neither blocks nor log segments found. Creating new log.
I20260925 13:09:58.803263 16681 log.cc:824] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:58.803666 16684 log.cc:824] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628: Log is configured to *not* fsync() on all Append() calls
I20260925 13:09:58.803722 16682 raft_consensus.cc:348] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } }
I20260925 13:09:58.803900 16682 raft_consensus.cc:374] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:58.803962 16682 raft_consensus.cc:729] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7099cb9663264a61bacd4a1c83989c26, State: Initialized, Role: FOLLOWER
I20260925 13:09:58.804085 16682 consensus_queue.cc:260] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } }
I20260925 13:09:58.804363 16682 ts_tablet_manager.cc:1428] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 13:09:58.804498 16530 heartbeater.cc:503] Master 127.0.206.126:36617 was elected leader, sending a full tablet report...
I20260925 13:09:58.804528 16684 tablet_bootstrap.cc:492] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628: No bootstrap required, opened a new log
I20260925 13:09:58.804589 16684 ts_tablet_manager.cc:1397] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 13:09:58.804647 16681 tablet_bootstrap.cc:492] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df: No bootstrap required, opened a new log
I20260925 13:09:58.804702 16681 ts_tablet_manager.cc:1397] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 13:09:58.806638 16684 raft_consensus.cc:348] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } }
I20260925 13:09:58.806981 16684 raft_consensus.cc:374] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:58.807015 16684 raft_consensus.cc:729] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2587f73a3c82462b8821e825761ef628, State: Initialized, Role: FOLLOWER
I20260925 13:09:58.806954 16681 raft_consensus.cc:348] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } }
I20260925 13:09:58.807106 16681 raft_consensus.cc:374] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:09:58.807128 16684 consensus_queue.cc:260] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } }
I20260925 13:09:58.807181 16681 raft_consensus.cc:729] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7418852b93f40139f3b93e08840d0df, State: Initialized, Role: FOLLOWER
I20260925 13:09:58.807312 16681 consensus_queue.cc:260] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } }
I20260925 13:09:58.807413 16684 ts_tablet_manager.cc:1428] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260925 13:09:58.807425 16396 heartbeater.cc:503] Master 127.0.206.126:36617 was elected leader, sending a full tablet report...
I20260925 13:09:58.807791 16681 ts_tablet_manager.cc:1428] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 13:09:58.807891 16664 heartbeater.cc:503] Master 127.0.206.126:36617 was elected leader, sending a full tablet report...
W20260925 13:09:58.859035 16531 tablet.cc:2369] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:09:58.943508 16397 tablet.cc:2369] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:09:59.008677 16665 tablet.cc:2369] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:09:59.031098 16689 raft_consensus.cc:482] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:09:59.031268 16689 raft_consensus.cc:504] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } }
I20260925 13:09:59.031653 16689 leader_election.cc:288] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7099cb9663264a61bacd4a1c83989c26 (127.0.206.66:38717), 2587f73a3c82462b8821e825761ef628 (127.0.206.65:40025)
I20260925 13:09:59.034433 16666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:59.031839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:40025 (local address 127.0.206.67:40565)
0925 13:09:59.031900 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:59.031913 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:59.031956 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:59.032127 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:59.032128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:59.032130 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:59.032206 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:59.032208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:59.032881 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:59.032882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:59.033641 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:59.033643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:59.034180 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:59.034181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:59.034264 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:59.034268 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:09:59.034325 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 13:09:59.034355 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":17}
I20260925 13:09:59.034456 16679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:59.031910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:40565 (local address 127.0.206.65:40025)
0925 13:09:59.031958 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:59.031960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:59.032009 (+ 49us) server_negotiation.cc:407] Connection header received
0925 13:09:59.032038 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:59.032039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:59.032049 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:59.032106 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:59.032291 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:59.032796 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:59.033741 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:59.034101 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:59.034160 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:59.034167 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:59.034341 (+ 174us) server_negotiation.cc:299] Negotiation successful
0925 13:09:59.034378 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 13:09:59.034545 16690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:59.031850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38717 (local address 127.0.206.67:39393)
0925 13:09:59.032010 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:59.032023 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:09:59.032073 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:09:59.032202 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:09:59.032202 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:09:59.032204 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:09:59.032305 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:59.032306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:59.032925 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:59.032926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:59.033662 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:09:59.033664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:09:59.034275 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:09:59.034276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:09:59.034362 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:59.034366 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:09:59.034422 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 13:09:59.034459 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":4,"thread_start_us":98,"threads_started":1}
I20260925 13:09:59.034566 16678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:59.032013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:39393 (local address 127.0.206.66:38717)
0925 13:09:59.032070 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:59.032072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:59.032083 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:09:59.032135 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:59.032136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:59.032145 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:59.032201 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:09:59.032376 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:59.032817 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:59.033784 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:59.034185 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:59.034254 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:59.034262 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:59.034448 (+ 186us) server_negotiation.cc:299] Negotiation successful
0925 13:09:59.034493 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 13:09:59.034878 16484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9636e736394410196a076b097aa3aff" candidate_uuid: "c7418852b93f40139f3b93e08840d0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7099cb9663264a61bacd4a1c83989c26" is_pre_election: true
I20260925 13:09:59.034900 16350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9636e736394410196a076b097aa3aff" candidate_uuid: "c7418852b93f40139f3b93e08840d0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2587f73a3c82462b8821e825761ef628" is_pre_election: true
I20260925 13:09:59.035036 16484 raft_consensus.cc:2454] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7418852b93f40139f3b93e08840d0df in term 0.
I20260925 13:09:59.035036 16350 raft_consensus.cc:2454] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7418852b93f40139f3b93e08840d0df in term 0.
I20260925 13:09:59.035266 16553 leader_election.cc:302] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2587f73a3c82462b8821e825761ef628, c7418852b93f40139f3b93e08840d0df; no voters:
I20260925 13:09:59.035383 16689 raft_consensus.cc:2781] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:09:59.035456 16689 raft_consensus.cc:482] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:09:59.035492 16689 raft_consensus.cc:3037] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:59.036165 16689 raft_consensus.cc:504] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } }
I20260925 13:09:59.036322 16689 leader_election.cc:288] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [CANDIDATE]: Term 1 election: Requested vote from peers 7099cb9663264a61bacd4a1c83989c26 (127.0.206.66:38717), 2587f73a3c82462b8821e825761ef628 (127.0.206.65:40025)
I20260925 13:09:59.036449 16484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9636e736394410196a076b097aa3aff" candidate_uuid: "c7418852b93f40139f3b93e08840d0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7099cb9663264a61bacd4a1c83989c26"
I20260925 13:09:59.036465 16350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9636e736394410196a076b097aa3aff" candidate_uuid: "c7418852b93f40139f3b93e08840d0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2587f73a3c82462b8821e825761ef628"
I20260925 13:09:59.036525 16484 raft_consensus.cc:3037] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:59.036551 16350 raft_consensus.cc:3037] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:09:59.037206 16484 raft_consensus.cc:2454] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7418852b93f40139f3b93e08840d0df in term 1.
I20260925 13:09:59.037297 16350 raft_consensus.cc:2454] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7418852b93f40139f3b93e08840d0df in term 1.
I20260925 13:09:59.037393 16552 leader_election.cc:302] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7099cb9663264a61bacd4a1c83989c26, c7418852b93f40139f3b93e08840d0df; no voters:
I20260925 13:09:59.037495 16689 raft_consensus.cc:2781] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:09:59.037633 16689 raft_consensus.cc:686] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [term 1 LEADER]: Becoming Leader. State: Replica: c7418852b93f40139f3b93e08840d0df, State: Running, Role: LEADER
I20260925 13:09:59.037742 16689 consensus_queue.cc:237] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } }
I20260925 13:09:59.038528 16204 catalog_manager.cc:5495] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df reported cstate change: term changed from 0 to 1, leader changed from <none> to c7418852b93f40139f3b93e08840d0df (127.0.206.67). New cstate: current_term: 1 leader_uuid: "c7418852b93f40139f3b93e08840d0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } health_report { overall_health: HEALTHY } } }
I20260925 13:09:59.074151 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:59.071716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60662 (local address 127.0.206.67:43321)
0925 13:09:59.071782 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 13:09:59.071785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:09:59.071795 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:09:59.071839 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:09:59.071840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:09:59.071857 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:09:59.071931 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:09:59.072113 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:59.072584 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:59.073354 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:09:59.073548 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:09:59.073625 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:09:59.073800 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:09:59.073900 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:09:59.074002 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 13:09:59.074027 (+ 25us) server_negotiation.cc:299] Negotiation successful
0925 13:09:59.074046 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 13:09:59.075814 16484 raft_consensus.cc:1261] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26 [term 1 FOLLOWER]: Refusing update from remote peer c7418852b93f40139f3b93e08840d0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:09:59.075853 16350 raft_consensus.cc:1261] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Refusing update from remote peer c7418852b93f40139f3b93e08840d0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:09:59.076159 16691 consensus_queue.cc:1035] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:09:59.076279 16689 consensus_queue.cc:1035] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:09:59.078727 16695 mvcc.cc:205] Tried to move back new op lower bound from 7333240009012215808 to 7333240008859078656. Current Snapshot: MvccSnapshot[applied={T|T < 7333240009012215808}]
I20260925 13:09:59.079353 16696 mvcc.cc:205] Tried to move back new op lower bound from 7333240009012215808 to 7333240008859078656. Current Snapshot: MvccSnapshot[applied={T|T < 7333240009012215808 or (T in {7333240009012215808})}]
I20260925 13:09:59.080543 16205 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:09:59.080639 16205 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:09:59.081343 16205 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: Sending DeleteTablet for 3 replicas of tablet a9636e736394410196a076b097aa3aff
I20260925 13:09:59.081667 16464 tablet_service.cc:1514] Processing DeleteTablet for tablet a9636e736394410196a076b097aa3aff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:09:59 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:32926
I20260925 13:09:59.081672 16330 tablet_service.cc:1514] Processing DeleteTablet for tablet a9636e736394410196a076b097aa3aff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:09:59 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:34836
I20260925 13:09:59.081897 16704 tablet_replica.cc:330] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26: stopping tablet replica
I20260925 13:09:59.081913 16598 tablet_service.cc:1514] Processing DeleteTablet for tablet a9636e736394410196a076b097aa3aff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:09:59 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:60646
I20260925 13:09:59.081995 16704 raft_consensus.cc:2229] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:09:59.082062 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 16267
I20260925 13:09:59.082109 16704 raft_consensus.cc:2258] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:09:59.083356 16704 ts_tablet_manager.cc:1905] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:09:59.085263 16704 ts_tablet_manager.cc:1918] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:09:59.085340 16704 log.cc:1196] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/a9636e736394410196a076b097aa3aff
I20260925 13:09:59.085582 16704 ts_tablet_manager.cc:1939] T a9636e736394410196a076b097aa3aff P 7099cb9663264a61bacd4a1c83989c26: Deleting consensus metadata
I20260925 13:09:59.085988 16190 catalog_manager.cc:4841] TS 7099cb9663264a61bacd4a1c83989c26 (127.0.206.66:38717): tablet a9636e736394410196a076b097aa3aff (table test-table [id=8a1102302c864961a51675867a13ea63]) successfully deleted
I20260925 13:09:59.086822 16706 tablet_replica.cc:330] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df: stopping tablet replica
I20260925 13:09:59.086923 16706 raft_consensus.cc:2229] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:09:59.087016 16706 raft_consensus.cc:2258] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:09:59.087384 16706 ts_tablet_manager.cc:1905] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:09:59.088712 16191 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:40025 (error 108)
I20260925 13:09:59.088886 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 16401
I20260925 13:09:59.089468 16677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:09:59.089019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:40025 (local address 127.0.0.1:34842)
0925 13:09:59.089393 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0925 13:09:59.089427 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:40025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357}
W20260925 13:09:59.090186 16191 catalog_manager.cc:4584] TS 2587f73a3c82462b8821e825761ef628 (127.0.206.65:40025): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a9636e736394410196a076b097aa3aff: Network error: Client connection negotiation failed: client connection to 127.0.206.65:40025: connect: Connection refused (error 111)
I20260925 13:09:59.091248 16706 ts_tablet_manager.cc:1918] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:09:59.091329 16706 log.cc:1196] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/a9636e736394410196a076b097aa3aff
I20260925 13:09:59.091581 16706 ts_tablet_manager.cc:1939] T a9636e736394410196a076b097aa3aff P c7418852b93f40139f3b93e08840d0df: Deleting consensus metadata
I20260925 13:09:59.091943 16189 catalog_manager.cc:4841] TS c7418852b93f40139f3b93e08840d0df (127.0.206.67:43321): tablet a9636e736394410196a076b097aa3aff (table test-table [id=8a1102302c864961a51675867a13ea63]) successfully deleted
I20260925 13:09:59.094384 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 16535
I20260925 13:09:59.099470 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 16173
I20260925 13:10:19.104753 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36617
--webserver_interface=127.0.206.126
--webserver_port=37719
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:36617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:10:19.184283 16708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:10:19.184461 16708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:10:19.184481 16708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:10:19.185633 16708 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 13:10:19.185672 16708 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 13:10:19.185686 16708 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 13:10:19.185698 16708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:10:19.185711 16708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:10:19.185724 16708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:10:19.185736 16708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:10:19.185748 16708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:10:19.187172 16708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:36617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=37719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:10:19.187342 16708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:10:19.187508 16708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:10:19.195868 16708 server_base.cc:1034] running on GCE node
W20260925 13:10:19.195874 16714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:19.195878 16717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:19.195972 16715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:19.196313 16708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:10:19.196620 16708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:10:19.197785 16708 hybrid_clock.cc:648] HybridClock initialized: now 1790341819197764 us; error 36 us; skew 500 ppm
Sep 25 13:10:19 dist-test-slave-chkf krb5kdc[16160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:10:19.200347 16708 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:10:19.200721 16708 webserver.cc:460] Webserver started at http://127.0.206.126:37719/ using document root <none> and password file <none>
I20260925 13:10:19.200901 16708 fs_manager.cc:359] Metadata directory not provided
I20260925 13:10:19.200951 16708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:10:19.202226 16708 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:10:19.202903 16724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:19.203084 16708 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:10:19.203190 16708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "98bf9d7fe1c044869a6a8394ab50df41"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "6dc2068df3a58354e5ddb0de00ab9764"
server_key_iv: "914680682f363c53b392182838596764"
server_key_version: "encryptionkey@0"
I20260925 13:10:19.203559 16708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:10:19.213445 16708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:10:19.215216 16708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:10:19.215368 16708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:10:19.219105 16708 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:36617
I20260925 13:10:19.219136 16776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:36617 every 8 connection(s)
I20260925 13:10:19.219551 16708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:10:19.220889 16777 sys_catalog.cc:263] Verifying existing consensus state
I20260925 13:10:19.221530 16777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: Bootstrap starting.
I20260925 13:10:19.229669 16777 log.cc:824] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: Log is configured to *not* fsync() on all Append() calls
I20260925 13:10:19.229718 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 16708
I20260925 13:10:19.230046 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:40025
--local_ip_for_outbound_sockets=127.0.206.65
--tserver_master_addrs=127.0.206.126:36617
--webserver_port=33403
--webserver_interface=127.0.206.65
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:10:19.231565 16777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:10:19.231853 16777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: Bootstrap complete.
I20260925 13:10:19.234066 16777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bf9d7fe1c044869a6a8394ab50df41" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36617 } }
I20260925 13:10:19.234308 16777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98bf9d7fe1c044869a6a8394ab50df41, State: Initialized, Role: FOLLOWER
I20260925 13:10:19.234511 16777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bf9d7fe1c044869a6a8394ab50df41" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36617 } }
I20260925 13:10:19.234607 16777 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:10:19.234664 16777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:10:19.234731 16777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 1 FOLLOWER]: Advancing to term 2
I20260925 13:10:19.235697 16777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bf9d7fe1c044869a6a8394ab50df41" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36617 } }
I20260925 13:10:19.235816 16777 leader_election.cc:302] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98bf9d7fe1c044869a6a8394ab50df41; no voters:
I20260925 13:10:19.236013 16777 leader_election.cc:288] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 13:10:19.236044 16781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 13:10:19.236153 16781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 2 LEADER]: Becoming Leader. State: Replica: 98bf9d7fe1c044869a6a8394ab50df41, State: Running, Role: LEADER
I20260925 13:10:19.236275 16777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:10:19.236292 16781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bf9d7fe1c044869a6a8394ab50df41" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36617 } }
I20260925 13:10:19.236678 16782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "98bf9d7fe1c044869a6a8394ab50df41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bf9d7fe1c044869a6a8394ab50df41" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36617 } } }
I20260925 13:10:19.236769 16782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [sys.catalog]: This master's current role is: LEADER
I20260925 13:10:19.237043 16783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98bf9d7fe1c044869a6a8394ab50df41. Latest consensus state: current_term: 2 leader_uuid: "98bf9d7fe1c044869a6a8394ab50df41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bf9d7fe1c044869a6a8394ab50df41" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36617 } } }
I20260925 13:10:19.237128 16783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [sys.catalog]: This master's current role is: LEADER
I20260925 13:10:19.237337 16789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:10:19.238199 16789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:10:19.238315 16789 catalog_manager.cc:1259] Loaded cluster ID: ff4631fa05b345f794bd3027117e9280
I20260925 13:10:19.238442 16789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:10:19.240020 16789 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:10:19.240226 16789 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: Loaded TSK: 0
I20260925 13:10:19.252094 16789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: Generated new TSK 1
I20260925 13:10:19.252175 16789 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 13:10:19.314327 16780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:10:19.314509 16780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:10:19.314532 16780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:10:19.315680 16780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:10:19.315721 16780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:10:19.315747 16780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:10:19.317126 16780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:40025
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=33403
--tserver_master_addrs=127.0.206.126:36617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:10:19.317337 16780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:10:19.317566 16780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:10:19.323468 16808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:19.323415 16805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:19.323428 16806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:19.323536 16780 server_base.cc:1034] running on GCE node
I20260925 13:10:19.323932 16780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:10:19.324204 16780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:10:19.325380 16780 hybrid_clock.cc:648] HybridClock initialized: now 1790341819325342 us; error 48 us; skew 500 ppm
Sep 25 13:10:19 dist-test-slave-chkf krb5kdc[16160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:10:19.328387 16780 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:10:19.328912 16780 webserver.cc:460] Webserver started at http://127.0.206.65:33403/ using document root <none> and password file <none>
I20260925 13:10:19.329138 16780 fs_manager.cc:359] Metadata directory not provided
I20260925 13:10:19.329216 16780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:10:19.330555 16780 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:10:19.331317 16815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:19.331532 16780 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 13:10:19.331638 16780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "2587f73a3c82462b8821e825761ef628"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "caa64392ce8701b60b5978a5cfd6569a"
server_key_iv: "498618e0fb6671293c960d3ff1f8be1b"
server_key_version: "encryptionkey@0"
I20260925 13:10:19.331969 16780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:10:19.352294 16780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:10:19.354346 16780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:10:19.354503 16780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:10:19.354797 16780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:10:19.355247 16822 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 13:10:19.356976 16780 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 13:10:19.357043 16780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 13:10:19.357091 16780 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 13:10:19.357717 16780 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 13:10:19.357774 16780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 13:10:19.357838 16822 tablet_bootstrap.cc:492] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628: Bootstrap starting.
I20260925 13:10:19.364727 16780 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:40025
I20260925 13:10:19.364804 16929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:40025 every 8 connection(s)
I20260925 13:10:19.365116 16780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:10:19.368664 16822 log.cc:824] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628: Log is configured to *not* fsync() on all Append() calls
I20260925 13:10:19.371111 16822 tablet_bootstrap.cc:492] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:10:19.371404 16822 tablet_bootstrap.cc:492] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628: Bootstrap complete.
I20260925 13:10:19.372566 16822 ts_tablet_manager.cc:1397] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
Sep 25 13:10:19 dist-test-slave-chkf krb5kdc[16160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:10:19.373597 16822 raft_consensus.cc:348] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } }
I20260925 13:10:19.373947 16822 raft_consensus.cc:729] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2587f73a3c82462b8821e825761ef628, State: Initialized, Role: FOLLOWER
I20260925 13:10:19.374115 16822 consensus_queue.cc:260] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } }
I20260925 13:10:19.374408 16822 ts_tablet_manager.cc:1428] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 13:10:19.375137 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 16780
I20260925 13:10:19.375427 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:38717
--local_ip_for_outbound_sockets=127.0.206.66
--tserver_master_addrs=127.0.206.126:36617
--webserver_port=34287
--webserver_interface=127.0.206.66
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:10:19.377100 16933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:19.365846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:56577 (local address 127.0.206.126:36617)
0925 13:10:19.366003 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:19.366012 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:19.366068 (+ 56us) server_negotiation.cc:407] Connection header received
0925 13:10:19.366951 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:19.366956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:19.366995 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:19.367089 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:19.368052 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.369065 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.370497 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.370958 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.371033 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:19.373359 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:19.373369 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:19.373369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:10:19.373378 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:19.375333 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:19.375851 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:19.375854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:19.375854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:19.375944 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:19.376199 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:19.376200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:19.376201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:19.376414 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:19.376511 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:19.376802 (+ 291us) server_negotiation.cc:299] Negotiation successful
0925 13:10:19.376925 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260925 13:10:19.377121 16932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:19.365745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:56577)
0925 13:10:19.365934 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:19.365977 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:19.366757 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:19.367181 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:19.367189 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:19.367623 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:10:19.367935 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:19.367944 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.369191 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.369192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:19.370400 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:19.370402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.371070 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.371071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:19.371207 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:19.371730 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:10:19.371740 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:10:19.373254 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:10:19.375428 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:19.375435 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:19.375437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:19.375755 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:19.376015 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:19.376016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:19.376017 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:19.376128 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:19.376488 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:10:19.376494 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:10:19.376628 (+ 134us) client_negotiation.cc:769] Sending connection context
0925 13:10:19.376735 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 13:10:19.376903 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":114,"threads_started":1}
I20260925 13:10:19.377849 16930 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36617
I20260925 13:10:19.377948 16930 heartbeater.cc:464] Registering TS with master...
I20260925 13:10:19.378120 16930 heartbeater.cc:511] Master 127.0.206.126:36617 requested a full tablet report, sending...
I20260925 13:10:19.378844 16741 ts_manager.cc:194] Registered new tserver with Master: 2587f73a3c82462b8821e825761ef628 (127.0.206.65:40025)
I20260925 13:10:19.379835 16741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:56577
I20260925 13:10:19.380754 16930 heartbeater.cc:503] Master 127.0.206.126:36617 was elected leader, sending a full tablet report...
Sep 25 13:10:19 dist-test-slave-chkf krb5kdc[16160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260925 13:10:19.388525 16938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:19.379454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52832 (local address 127.0.206.65:40025)
0925 13:10:19.379611 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:19.379615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:19.379790 (+ 175us) server_negotiation.cc:407] Connection header received
0925 13:10:19.379937 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:19.379939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:19.379954 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:19.379999 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:19.381464 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.382075 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.383274 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.383534 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.383588 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:19.385659 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:19.385666 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:19.385666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:10:19.385673 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:19.387245 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:19.387695 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:19.387702 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:19.387702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:19.387757 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:19.387957 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:19.387958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:19.387958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:19.388086 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:19.388181 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:19.388348 (+ 167us) server_negotiation.cc:299] Negotiation successful
0925 13:10:19.388431 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260925 13:10:19.388525 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:19.379389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:40025 (local address 127.0.0.1:52832)
0925 13:10:19.379686 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:19.379726 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:19.379829 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:19.380187 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:19.380190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:19.381315 (+ 1125us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:10:19.381379 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:19.381381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.382160 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.382161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:19.383193 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:19.383196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.383612 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.383613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:19.383700 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:19.384105 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:10:19.384111 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:10:19.385574 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:10:19.387346 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:19.387352 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:19.387354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:19.387593 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:19.387825 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:19.387825 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:19.387826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:19.387885 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:19.388156 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:10:19.388162 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:10:19.388210 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 13:10:19.388314 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 13:10:19.388430 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":78,"threads_started":1}
I20260925 13:10:19.388911 16864 tablet_service.cc:1514] Processing DeleteTablet for tablet a9636e736394410196a076b097aa3aff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:09:59 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:52832
I20260925 13:10:19.389448 16939 tablet_replica.cc:330] stopping tablet replica
I20260925 13:10:19.389564 16939 raft_consensus.cc:2229] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:10:19.389626 16939 raft_consensus.cc:2258] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:10:19.389897 16939 ts_tablet_manager.cc:1905] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:10:19.392299 16939 ts_tablet_manager.cc:1918] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:10:19.392370 16939 log.cc:1196] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/wal/wals/a9636e736394410196a076b097aa3aff
I20260925 13:10:19.392603 16939 ts_tablet_manager.cc:1939] T a9636e736394410196a076b097aa3aff P 2587f73a3c82462b8821e825761ef628: Deleting consensus metadata
I20260925 13:10:19.393152 16727 catalog_manager.cc:4841] TS 2587f73a3c82462b8821e825761ef628 (127.0.206.65:40025): tablet a9636e736394410196a076b097aa3aff (table test-table [id=8a1102302c864961a51675867a13ea63]) successfully deleted
W20260925 13:10:19.459856 16936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:10:19.460036 16936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:10:19.460059 16936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:10:19.461198 16936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:10:19.461241 16936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:10:19.461266 16936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:10:19.462949 16936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:38717
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=34287
--tserver_master_addrs=127.0.206.126:36617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:10:19.463179 16936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:10:19.463387 16936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:10:19.468441 16936 server_base.cc:1034] running on GCE node
W20260925 13:10:19.468554 16945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:19.468567 16947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:19.468768 16944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:19.468968 16936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:10:19.469197 16936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:10:19.470350 16936 hybrid_clock.cc:648] HybridClock initialized: now 1790341819470335 us; error 25 us; skew 500 ppm
Sep 25 13:10:19 dist-test-slave-chkf krb5kdc[16160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:10:19.472733 16936 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:10:19.473155 16936 webserver.cc:460] Webserver started at http://127.0.206.66:34287/ using document root <none> and password file <none>
I20260925 13:10:19.473361 16936 fs_manager.cc:359] Metadata directory not provided
I20260925 13:10:19.473438 16936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:10:19.474656 16936 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:10:19.475235 16954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:19.475400 16936 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:19.475497 16936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "7099cb9663264a61bacd4a1c83989c26"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "6946de8ff053c97ddf4535605427b6c7"
server_key_iv: "8cecc3d4ba7b9583a8754a51c4cf7859"
server_key_version: "encryptionkey@0"
I20260925 13:10:19.475811 16936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:10:19.489705 16936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:10:19.491479 16936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:10:19.491631 16936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:10:19.491844 16936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:10:19.492128 16936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:10:19.492180 16936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:19.492214 16936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:10:19.492242 16936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:19.497792 16936 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:38717
I20260925 13:10:19.497841 17067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:38717 every 8 connection(s)
I20260925 13:10:19.498143 16936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:10:19.500771 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 16936
I20260925 13:10:19.501067 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:43321
--local_ip_for_outbound_sockets=127.0.206.67
--tserver_master_addrs=127.0.206.126:36617
--webserver_port=45995
--webserver_interface=127.0.206.67
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 13:10:19 dist-test-slave-chkf krb5kdc[16160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:10:19.508555 16933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:19.498775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:60651 (local address 127.0.206.126:36617)
0925 13:10:19.498827 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:19.498830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:19.499062 (+ 232us) server_negotiation.cc:407] Connection header received
0925 13:10:19.499541 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:19.499542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:19.499552 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:19.499597 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:19.500445 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.501211 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.502733 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.503096 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.503175 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:19.505564 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:19.505566 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:19.505566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:10:19.505590 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:19.507094 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:19.507627 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:19.507628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:19.507628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:19.507675 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:19.507946 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:19.507947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:19.507947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:19.508042 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:19.508089 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:19.508423 (+ 334us) server_negotiation.cc:299] Negotiation successful
0925 13:10:19.508486 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 13:10:19.508841 17070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:19.498773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:60651)
0925 13:10:19.498913 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:19.498964 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:19.499440 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:19.499657 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:19.499660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:19.500197 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:10:19.500372 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:19.500377 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.501319 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.501321 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:19.502622 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:19.502625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.503182 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.503183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:19.503276 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:19.503759 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:10:19.503767 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:10:19.505465 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:10:19.507171 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:19.507179 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:19.507181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:19.507549 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:19.507733 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:19.507734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:19.507735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:19.507874 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:19.508105 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:10:19.508111 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:10:19.508278 (+ 167us) client_negotiation.cc:769] Sending connection context
0925 13:10:19.508399 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 13:10:19.508573 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260925 13:10:19.509334 17068 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36617
I20260925 13:10:19.509421 17068 heartbeater.cc:464] Registering TS with master...
I20260925 13:10:19.509605 17068 heartbeater.cc:511] Master 127.0.206.126:36617 requested a full tablet report, sending...
I20260925 13:10:19.509968 16741 ts_manager.cc:194] Registered new tserver with Master: 7099cb9663264a61bacd4a1c83989c26 (127.0.206.66:38717)
I20260925 13:10:19.510285 16741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:60651
W20260925 13:10:19.581390 17071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:10:19.581601 17071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:10:19.581640 17071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:10:19.582758 17071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:10:19.582820 17071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:10:19.582865 17071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:10:19.584302 17071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:43321
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=45995
--tserver_master_addrs=127.0.206.126:36617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:10:19.584550 17071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:10:19.584762 17071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:10:19.589601 17076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:19.589589 17077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:19.589576 17079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:19.589854 17071 server_base.cc:1034] running on GCE node
I20260925 13:10:19.590034 17071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:10:19.590257 17071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:10:19.591410 17071 hybrid_clock.cc:648] HybridClock initialized: now 1790341819591396 us; error 26 us; skew 500 ppm
Sep 25 13:10:19 dist-test-slave-chkf krb5kdc[16160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:10:19.593745 17071 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:10:19.594092 17071 webserver.cc:460] Webserver started at http://127.0.206.67:45995/ using document root <none> and password file <none>
I20260925 13:10:19.594256 17071 fs_manager.cc:359] Metadata directory not provided
I20260925 13:10:19.594307 17071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:10:19.595543 17071 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:10:19.596125 17086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:19.596290 17071 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:10:19.596365 17071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "c7418852b93f40139f3b93e08840d0df"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "4c65fa7b538a7bddb868277065beb27d"
server_key_iv: "9bdb79ffcbabff4b94445dfaf3cb9c59"
server_key_version: "encryptionkey@0"
I20260925 13:10:19.596632 17071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:10:19.613485 17071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:10:19.615376 17071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:10:19.615489 17071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:10:19.615643 17071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:10:19.615981 17071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:10:19.616022 17071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:19.616042 17071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:10:19.616082 17071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:19.621364 17071 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:43321
I20260925 13:10:19.621479 17199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:43321 every 8 connection(s)
I20260925 13:10:19.621747 17071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:10:19.626151 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 17071
Sep 25 13:10:19 dist-test-slave-chkf krb5kdc[16160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:10:19.630795 16933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:19.622426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:40229 (local address 127.0.206.126:36617)
0925 13:10:19.622507 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:19.622510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:19.622685 (+ 175us) server_negotiation.cc:407] Connection header received
0925 13:10:19.623150 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:19.623152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:19.623162 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:19.623213 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:19.624050 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.624514 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.625435 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.625631 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.625695 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:19.628003 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:19.628005 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:19.628012 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:10:19.628020 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:19.629493 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:19.629853 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:19.629855 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:19.629855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:19.629918 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:19.630153 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:19.630155 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:19.630155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:19.630267 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:19.630344 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:19.630585 (+ 241us) server_negotiation.cc:299] Negotiation successful
0925 13:10:19.630663 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 13:10:19.630983 17202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:19.622387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:40229)
0925 13:10:19.622567 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:19.622598 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:19.623067 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:19.623274 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:19.623277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:19.623808 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:10:19.623977 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:19.623981 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.624593 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.624594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:19.625343 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:19.625345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.625711 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.625712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:19.625817 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:19.626321 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:10:19.626329 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:10:19.627904 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:10:19.629571 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:19.629576 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:19.629578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:19.629779 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:19.629984 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:19.629985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:19.629985 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:19.630081 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:19.630335 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:10:19.630340 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:10:19.630457 (+ 117us) client_negotiation.cc:769] Sending connection context
0925 13:10:19.630563 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 13:10:19.630708 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
I20260925 13:10:19.631574 17200 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36617
I20260925 13:10:19.631686 17200 heartbeater.cc:464] Registering TS with master...
I20260925 13:10:19.631883 17200 heartbeater.cc:511] Master 127.0.206.126:36617 requested a full tablet report, sending...
I20260925 13:10:19.632254 16741 ts_manager.cc:194] Registered new tserver with Master: c7418852b93f40139f3b93e08840d0df (127.0.206.67:43321)
I20260925 13:10:19.632630 16741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:40229
I20260925 13:10:19.633636 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:19.626639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43710 (local address 127.0.206.126:36617)
0925 13:10:19.626767 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:19.626769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:19.626810 (+ 41us) server_negotiation.cc:407] Connection header received
0925 13:10:19.626881 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:19.626882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:19.626895 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:19.626967 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:19.627580 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.628092 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.629189 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.629372 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.629419 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:19.630671 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:19.630672 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:19.630673 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:10:19.630679 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:19.632496 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:19.632893 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:19.632895 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:19.632895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:19.632949 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:19.633152 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:19.633153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:19.633153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:19.633258 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:19.633428 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:19.633465 (+ 37us) server_negotiation.cc:299] Negotiation successful
0925 13:10:19.633537 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":102,"threads_started":1}
I20260925 13:10:19.634305 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:10:19.636803 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:19.634610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43714 (local address 127.0.206.126:36617)
0925 13:10:19.634669 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:19.634671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:19.634773 (+ 102us) server_negotiation.cc:407] Connection header received
0925 13:10:19.634855 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:19.634856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:19.634880 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:19.634937 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:10:19.635005 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.635423 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.636215 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.636473 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.636517 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:19.636638 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:10:19.636671 (+ 33us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0925 13:10:19.636767 (+ 96us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43714: token expired
Metrics: {"server-negotiator.queue_time_us":37}
W20260925 13:10:19.636874 17204 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43714: token expired
W20260925 13:10:19.636968 16667 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.206.126:36617)
I20260925 13:10:19.641912 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:19.637171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43716 (local address 127.0.206.126:36617)
0925 13:10:19.637227 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:19.637228 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:19.637236 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:10:19.637255 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:19.637256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:19.637265 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:19.637324 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:19.637769 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.638177 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.638832 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.638993 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.639050 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:19.639967 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:19.639968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:19.639969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:10:19.639974 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:19.641088 (+ 1114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:19.641358 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:19.641359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:19.641359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:19.641401 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:19.641561 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:19.641561 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:19.641561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:19.641635 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:19.641696 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:19.641776 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 13:10:19.641827 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 13:10:19.642874 16741 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43716:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:10:19.644320 16741 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:10:19.646234 16864 tablet_service.cc:1467] Processing CreateTablet for tablet 596188104c974cd0a7e35fc9fc5bf5d9 (DEFAULT_TABLE table=test-table [id=d7dd5ca966d2452c9efd7f880e758622]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:10:19.646418 16864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 596188104c974cd0a7e35fc9fc5bf5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:10:19.648020 16822 tablet_bootstrap.cc:492] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628: Bootstrap starting.
I20260925 13:10:19.648334 16822 tablet_bootstrap.cc:654] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628: Neither blocks nor log segments found. Creating new log.
I20260925 13:10:19.648852 16822 tablet_bootstrap.cc:492] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628: No bootstrap required, opened a new log
I20260925 13:10:19.648917 16822 ts_tablet_manager.cc:1397] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260925 13:10:19.649226 16822 raft_consensus.cc:348] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } }
I20260925 13:10:19.649303 16822 raft_consensus.cc:374] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:10:19.649351 16822 raft_consensus.cc:729] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2587f73a3c82462b8821e825761ef628, State: Initialized, Role: FOLLOWER
I20260925 13:10:19.649399 16822 consensus_queue.cc:260] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } }
I20260925 13:10:19.649499 16822 ts_tablet_manager.cc:1428] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 13:10:19.650290 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:19.646008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:43321 (local address 127.0.0.1:58838)
0925 13:10:19.646175 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:19.646183 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:19.646217 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:19.647106 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:19.647107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:19.647109 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:10:19.647143 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:19.647144 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.648285 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.648286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:19.649223 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:19.649226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.650010 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.650011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:19.650070 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:19.650090 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 13:10:19.650178 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 13:10:19.650220 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":10,"thread_start_us":111,"threads_started":1}
I20260925 13:10:19.650314 17208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:19.646772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58838 (local address 127.0.206.67:43321)
0925 13:10:19.646941 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:19.646945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:19.646956 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:10:19.646986 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:19.646988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:19.647003 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:19.647095 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:10:19.647213 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.648210 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.649320 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.649907 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.650003 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:19.650066 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:19.650201 (+ 135us) server_negotiation.cc:299] Negotiation successful
0925 13:10:19.650243 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260925 13:10:19.650609 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:19.646008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38717 (local address 127.0.0.1:60050)
0925 13:10:19.646052 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:19.646062 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:19.646098 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:19.646516 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:19.646517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:19.646518 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:10:19.646572 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:19.646573 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.647600 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.647601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:19.648551 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:19.648554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.650422 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.650423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:19.650509 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:19.650512 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:10:19.650540 (+ 28us) client_negotiation.cc:240] Negotiation successful
0925 13:10:19.650569 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 13:10:19.650976 17207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:19.646159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60050 (local address 127.0.206.66:38717)
0925 13:10:19.646351 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:19.646355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:19.646365 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:10:19.646401 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:19.646402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:19.646423 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:19.646513 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:10:19.646630 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.647525 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.648697 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:19.650318 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:19.650754 (+ 436us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:19.650818 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:19.650849 (+ 31us) server_negotiation.cc:299] Negotiation successful
0925 13:10:19.650905 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":98,"threads_started":1}
I20260925 13:10:19.651259 17134 tablet_service.cc:1467] Processing CreateTablet for tablet 596188104c974cd0a7e35fc9fc5bf5d9 (DEFAULT_TABLE table=test-table [id=d7dd5ca966d2452c9efd7f880e758622]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:10:19.651582 17134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 596188104c974cd0a7e35fc9fc5bf5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:10:19.651602 17002 tablet_service.cc:1467] Processing CreateTablet for tablet 596188104c974cd0a7e35fc9fc5bf5d9 (DEFAULT_TABLE table=test-table [id=d7dd5ca966d2452c9efd7f880e758622]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:10:19.651919 17002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 596188104c974cd0a7e35fc9fc5bf5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:10:19.654304 17210 tablet_bootstrap.cc:492] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df: Bootstrap starting.
I20260925 13:10:19.654757 17211 tablet_bootstrap.cc:492] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26: Bootstrap starting.
I20260925 13:10:19.654969 17210 tablet_bootstrap.cc:654] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df: Neither blocks nor log segments found. Creating new log.
I20260925 13:10:19.655313 17211 tablet_bootstrap.cc:654] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26: Neither blocks nor log segments found. Creating new log.
I20260925 13:10:19.655467 17210 log.cc:824] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df: Log is configured to *not* fsync() on all Append() calls
I20260925 13:10:19.655840 17211 log.cc:824] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26: Log is configured to *not* fsync() on all Append() calls
I20260925 13:10:19.656282 17210 tablet_bootstrap.cc:492] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df: No bootstrap required, opened a new log
I20260925 13:10:19.656374 17210 ts_tablet_manager.cc:1397] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:10:19.656576 17211 tablet_bootstrap.cc:492] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26: No bootstrap required, opened a new log
I20260925 13:10:19.656654 17211 ts_tablet_manager.cc:1397] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:10:19.657698 17210 raft_consensus.cc:348] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } }
I20260925 13:10:19.657822 17210 raft_consensus.cc:374] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:10:19.657843 17210 raft_consensus.cc:729] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7418852b93f40139f3b93e08840d0df, State: Initialized, Role: FOLLOWER
I20260925 13:10:19.657960 17210 consensus_queue.cc:260] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } }
I20260925 13:10:19.657985 17211 raft_consensus.cc:348] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } }
I20260925 13:10:19.658166 17211 raft_consensus.cc:374] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:10:19.658209 17211 raft_consensus.cc:729] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7099cb9663264a61bacd4a1c83989c26, State: Initialized, Role: FOLLOWER
I20260925 13:10:19.658222 17200 heartbeater.cc:503] Master 127.0.206.126:36617 was elected leader, sending a full tablet report...
I20260925 13:10:19.658238 17210 ts_tablet_manager.cc:1428] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:10:19.658351 17211 consensus_queue.cc:260] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } }
I20260925 13:10:19.658608 17068 heartbeater.cc:503] Master 127.0.206.126:36617 was elected leader, sending a full tablet report...
I20260925 13:10:19.658613 17211 ts_tablet_manager.cc:1428] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260925 13:10:19.748680 17069 tablet.cc:2369] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:10:19.865883 16931 tablet.cc:2369] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:10:19.872323 17201 tablet.cc:2369] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:10:20.031502 16935 raft_consensus.cc:482] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:10:20.031678 16935 raft_consensus.cc:504] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } }
I20260925 13:10:20.032107 16935 leader_election.cc:288] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7099cb9663264a61bacd4a1c83989c26 (127.0.206.66:38717), c7418852b93f40139f3b93e08840d0df (127.0.206.67:43321)
I20260925 13:10:20.035277 17207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:20.032466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:39961 (local address 127.0.206.66:38717)
0925 13:10:20.032529 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:20.032532 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:20.032643 (+ 111us) server_negotiation.cc:407] Connection header received
0925 13:10:20.032742 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:20.032743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:20.032751 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:20.032821 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:10:20.032996 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:20.033434 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:20.034446 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:20.034922 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:20.035020 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:20.035028 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:20.035169 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 13:10:20.035206 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 13:10:20.035279 17208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:20.032446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:49459 (local address 127.0.206.67:43321)
0925 13:10:20.032519 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:20.032521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:20.032605 (+ 84us) server_negotiation.cc:407] Connection header received
0925 13:10:20.032721 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:20.032722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:20.032733 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:20.032788 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:10:20.032944 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:20.033424 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:20.034508 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:20.034919 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:20.034977 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:20.035005 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:20.035176 (+ 171us) server_negotiation.cc:299] Negotiation successful
0925 13:10:20.035211 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 13:10:20.035297 17216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:20.032402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38717 (local address 127.0.206.65:39961)
0925 13:10:20.032540 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:20.032550 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:20.032643 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:20.032839 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:20.032840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:20.032842 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:10:20.032889 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:20.032891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:20.033552 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:20.033553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:20.034353 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:20.034355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:20.035033 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:20.035034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:20.035094 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:20.035097 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:10:20.035163 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 13:10:20.035205 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":26,"thread_start_us":56,"threads_started":1}
I20260925 13:10:20.035336 17217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:20.032333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:43321 (local address 127.0.206.65:49459)
0925 13:10:20.032525 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:20.032538 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:20.032636 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:20.032812 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:20.032813 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:20.032815 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:10:20.032874 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:20.032876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:20.033520 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:20.033521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:20.034356 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:20.034358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:20.035000 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:20.035001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:20.035062 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:20.035084 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 13:10:20.035162 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 13:10:20.035207 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":48,"thread_start_us":73,"threads_started":1}
I20260925 13:10:20.035641 17022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "596188104c974cd0a7e35fc9fc5bf5d9" candidate_uuid: "2587f73a3c82462b8821e825761ef628" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7099cb9663264a61bacd4a1c83989c26" is_pre_election: true
I20260925 13:10:20.035812 17022 raft_consensus.cc:2454] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2587f73a3c82462b8821e825761ef628 in term 0.
I20260925 13:10:20.035837 17154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "596188104c974cd0a7e35fc9fc5bf5d9" candidate_uuid: "2587f73a3c82462b8821e825761ef628" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7418852b93f40139f3b93e08840d0df" is_pre_election: true
I20260925 13:10:20.036001 17154 raft_consensus.cc:2454] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2587f73a3c82462b8821e825761ef628 in term 0.
I20260925 13:10:20.036041 16817 leader_election.cc:302] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2587f73a3c82462b8821e825761ef628, 7099cb9663264a61bacd4a1c83989c26; no voters:
I20260925 13:10:20.036154 16935 raft_consensus.cc:2781] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:10:20.036253 16935 raft_consensus.cc:482] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:10:20.036283 16935 raft_consensus.cc:3037] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:10:20.036954 16935 raft_consensus.cc:504] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } }
I20260925 13:10:20.037120 16935 leader_election.cc:288] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 1 election: Requested vote from peers 7099cb9663264a61bacd4a1c83989c26 (127.0.206.66:38717), c7418852b93f40139f3b93e08840d0df (127.0.206.67:43321)
I20260925 13:10:20.037256 17022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "596188104c974cd0a7e35fc9fc5bf5d9" candidate_uuid: "2587f73a3c82462b8821e825761ef628" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7099cb9663264a61bacd4a1c83989c26"
I20260925 13:10:20.037295 17154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "596188104c974cd0a7e35fc9fc5bf5d9" candidate_uuid: "2587f73a3c82462b8821e825761ef628" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7418852b93f40139f3b93e08840d0df"
I20260925 13:10:20.037340 17022 raft_consensus.cc:3037] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:10:20.037405 17154 raft_consensus.cc:3037] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:10:20.037983 17022 raft_consensus.cc:2454] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2587f73a3c82462b8821e825761ef628 in term 1.
I20260925 13:10:20.038168 16817 leader_election.cc:302] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2587f73a3c82462b8821e825761ef628, 7099cb9663264a61bacd4a1c83989c26; no voters:
I20260925 13:10:20.038261 16935 raft_consensus.cc:2781] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:10:20.038305 17154 raft_consensus.cc:2454] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2587f73a3c82462b8821e825761ef628 in term 1.
I20260925 13:10:20.038369 16935 raft_consensus.cc:686] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 LEADER]: Becoming Leader. State: Replica: 2587f73a3c82462b8821e825761ef628, State: Running, Role: LEADER
I20260925 13:10:20.038440 16935 consensus_queue.cc:237] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } }
I20260925 13:10:20.039207 16740 catalog_manager.cc:5495] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2587f73a3c82462b8821e825761ef628 (127.0.206.65). New cstate: current_term: 1 leader_uuid: "2587f73a3c82462b8821e825761ef628" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } health_report { overall_health: HEALTHY } } }
I20260925 13:10:20.091216 17221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:20.088726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52840 (local address 127.0.206.65:40025)
0925 13:10:20.088859 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:20.088861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:20.088867 (+ 6us) server_negotiation.cc:407] Connection header received
0925 13:10:20.088904 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:20.088905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:20.088923 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:20.089024 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:10:20.089136 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:20.089634 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:20.090370 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:20.090591 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:20.090668 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:20.090859 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:10:20.090963 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:10:20.091055 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:20.091099 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 13:10:20.091119 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260925 13:10:20.092777 17022 raft_consensus.cc:1261] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26 [term 1 FOLLOWER]: Refusing update from remote peer 2587f73a3c82462b8821e825761ef628: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:10:20.092800 17154 raft_consensus.cc:1261] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df [term 1 FOLLOWER]: Refusing update from remote peer 2587f73a3c82462b8821e825761ef628: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:10:20.093133 16935 consensus_queue.cc:1035] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:10:20.093291 17218 consensus_queue.cc:1035] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:10:20.095811 17223 mvcc.cc:205] Tried to move back new op lower bound from 7333240095097516032 to 7333240094877933568. Current Snapshot: MvccSnapshot[applied={T|T < 7333240095097516032}]
I20260925 13:10:20.097370 17226 mvcc.cc:205] Tried to move back new op lower bound from 7333240095097516032 to 7333240094877933568. Current Snapshot: MvccSnapshot[applied={T|T < 7333240095097516032}]
I20260925 13:10:20.097805 16741 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:10:20.097898 16741 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:10:20.098568 16741 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: Sending DeleteTablet for 3 replicas of tablet 596188104c974cd0a7e35fc9fc5bf5d9
I20260925 13:10:20.098878 16864 tablet_service.cc:1514] Processing DeleteTablet for tablet 596188104c974cd0a7e35fc9fc5bf5d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:10:20 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:52832
I20260925 13:10:20.098940 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 16780
I20260925 13:10:20.099532 17002 tablet_service.cc:1514] Processing DeleteTablet for tablet 596188104c974cd0a7e35fc9fc5bf5d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:10:20 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:60050
I20260925 13:10:20.100510 17232 tablet_replica.cc:330] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26: stopping tablet replica
I20260925 13:10:20.100522 17134 tablet_service.cc:1514] Processing DeleteTablet for tablet 596188104c974cd0a7e35fc9fc5bf5d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:10:20 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:58838
I20260925 13:10:20.100621 17232 raft_consensus.cc:2229] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:10:20.100754 17232 raft_consensus.cc:2258] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:10:20.100916 17233 tablet_replica.cc:330] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df: stopping tablet replica
I20260925 13:10:20.101006 17233 raft_consensus.cc:2229] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:10:20.101094 17233 raft_consensus.cc:2258] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:10:20.101161 17232 ts_tablet_manager.cc:1905] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:10:20.101574 17233 ts_tablet_manager.cc:1905] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:10:20.102854 17232 ts_tablet_manager.cc:1918] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:10:20.102913 17232 log.cc:1196] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/596188104c974cd0a7e35fc9fc5bf5d9
I20260925 13:10:20.103179 17233 ts_tablet_manager.cc:1918] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:10:20.103212 17232 ts_tablet_manager.cc:1939] T 596188104c974cd0a7e35fc9fc5bf5d9 P 7099cb9663264a61bacd4a1c83989c26: Deleting consensus metadata
I20260925 13:10:20.103238 17233 log.cc:1196] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/596188104c974cd0a7e35fc9fc5bf5d9
I20260925 13:10:20.103493 17233 ts_tablet_manager.cc:1939] T 596188104c974cd0a7e35fc9fc5bf5d9 P c7418852b93f40139f3b93e08840d0df: Deleting consensus metadata
I20260925 13:10:20.103922 16726 catalog_manager.cc:4841] TS 7099cb9663264a61bacd4a1c83989c26 (127.0.206.66:38717): tablet 596188104c974cd0a7e35fc9fc5bf5d9 (table test-table [id=d7dd5ca966d2452c9efd7f880e758622]) successfully deleted
I20260925 13:10:20.104087 16725 catalog_manager.cc:4841] TS c7418852b93f40139f3b93e08840d0df (127.0.206.67:43321): tablet 596188104c974cd0a7e35fc9fc5bf5d9 (table test-table [id=d7dd5ca966d2452c9efd7f880e758622]) successfully deleted
W20260925 13:10:20.104885 16727 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:40025 (error 108)
I20260925 13:10:20.105115 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 16936
I20260925 13:10:20.106026 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:20.105285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:40025 (local address 127.0.0.1:52842)
0925 13:10:20.105945 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:20.106001 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:40025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":640}
W20260925 13:10:20.106153 16727 catalog_manager.cc:4584] TS 2587f73a3c82462b8821e825761ef628 (127.0.206.65:40025): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 596188104c974cd0a7e35fc9fc5bf5d9: Network error: Client connection negotiation failed: client connection to 127.0.206.65:40025: connect: Connection refused (error 111)
I20260925 13:10:20.110453 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 17071
I20260925 13:10:20.115268 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 16708
I20260925 13:10:40.120647 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36617
--webserver_interface=127.0.206.126
--webserver_port=37719
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:36617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:10:40.200227 17235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:10:40.200438 17235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:10:40.200460 17235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:10:40.201642 17235 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 13:10:40.201684 17235 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 13:10:40.201697 17235 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 13:10:40.201709 17235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:10:40.201766 17235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:10:40.201783 17235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:10:40.201807 17235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:10:40.201830 17235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:10:40.203330 17235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:36617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=37719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:10:40.203601 17235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:10:40.203860 17235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:10:40.208691 17240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:40.208705 17243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:40.208788 17241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:40.208791 17235 server_base.cc:1034] running on GCE node
I20260925 13:10:40.209139 17235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:10:40.209467 17235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:10:40.210608 17235 hybrid_clock.cc:648] HybridClock initialized: now 1790341840210592 us; error 41 us; skew 500 ppm
Sep 25 13:10:40 dist-test-slave-chkf krb5kdc[16160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341840, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:10:40.213389 17235 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:10:40.213855 17235 webserver.cc:460] Webserver started at http://127.0.206.126:37719/ using document root <none> and password file <none>
I20260925 13:10:40.214102 17235 fs_manager.cc:359] Metadata directory not provided
I20260925 13:10:40.214149 17235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:10:40.215512 17235 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:10:40.216161 17250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:40.216336 17235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:10:40.216423 17235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "98bf9d7fe1c044869a6a8394ab50df41"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "6dc2068df3a58354e5ddb0de00ab9764"
server_key_iv: "914680682f363c53b392182838596764"
server_key_version: "encryptionkey@0"
I20260925 13:10:40.216763 17235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:10:40.234668 17235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:10:40.237126 17235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:10:40.237308 17235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:10:40.241405 17235 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:36617
I20260925 13:10:40.241441 17302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:36617 every 8 connection(s)
I20260925 13:10:40.241834 17235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:10:40.243363 17303 sys_catalog.cc:263] Verifying existing consensus state
I20260925 13:10:40.243937 17303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: Bootstrap starting.
I20260925 13:10:40.245662 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 17235
I20260925 13:10:40.245963 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:40025
--local_ip_for_outbound_sockets=127.0.206.65
--tserver_master_addrs=127.0.206.126:36617
--webserver_port=33403
--webserver_interface=127.0.206.65
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:10:40.252714 17303 log.cc:824] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: Log is configured to *not* fsync() on all Append() calls
I20260925 13:10:40.254971 17303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:10:40.255272 17303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: Bootstrap complete.
I20260925 13:10:40.257490 17303 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bf9d7fe1c044869a6a8394ab50df41" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36617 } }
I20260925 13:10:40.257709 17303 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98bf9d7fe1c044869a6a8394ab50df41, State: Initialized, Role: FOLLOWER
I20260925 13:10:40.257882 17303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bf9d7fe1c044869a6a8394ab50df41" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36617 } }
I20260925 13:10:40.257939 17303 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:10:40.258002 17303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:10:40.258050 17303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 2 FOLLOWER]: Advancing to term 3
I20260925 13:10:40.259116 17303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bf9d7fe1c044869a6a8394ab50df41" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36617 } }
I20260925 13:10:40.259202 17303 leader_election.cc:302] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98bf9d7fe1c044869a6a8394ab50df41; no voters:
I20260925 13:10:40.259456 17303 leader_election.cc:288] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 13:10:40.259487 17307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 13:10:40.259603 17307 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [term 3 LEADER]: Becoming Leader. State: Replica: 98bf9d7fe1c044869a6a8394ab50df41, State: Running, Role: LEADER
I20260925 13:10:40.259737 17303 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:10:40.259744 17307 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bf9d7fe1c044869a6a8394ab50df41" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36617 } }
I20260925 13:10:40.260138 17308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "98bf9d7fe1c044869a6a8394ab50df41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bf9d7fe1c044869a6a8394ab50df41" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36617 } } }
I20260925 13:10:40.260154 17309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98bf9d7fe1c044869a6a8394ab50df41. Latest consensus state: current_term: 3 leader_uuid: "98bf9d7fe1c044869a6a8394ab50df41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bf9d7fe1c044869a6a8394ab50df41" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36617 } } }
I20260925 13:10:40.260452 17308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [sys.catalog]: This master's current role is: LEADER
I20260925 13:10:40.260462 17309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41 [sys.catalog]: This master's current role is: LEADER
I20260925 13:10:40.261682 17323 catalog_manager.cc:1259] Loaded cluster ID: ff4631fa05b345f794bd3027117e9280
I20260925 13:10:40.261729 17323 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: loading cluster ID for follower catalog manager: success
I20260925 13:10:40.262900 17323 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 98bf9d7fe1c044869a6a8394ab50df41: acquiring CA information for follower catalog manager: success
F20260925 13:10:40.263283 17323 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9e6d77b9ad google::LogMessage::Fail() at ??:0
@ 0x7f9e6d77e88a google::LogMessage::SendToLog() at ??:0
@ 0x7f9e6d77b4de google::LogMessage::Flush() at ??:0
@ 0x7f9e6d77ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9e6fee8104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f9e756d46b9 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f9e756d4989 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f9e756d4e4e kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f9e756d78c2 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f9e6f63e954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9e704a26db start_thread at ??:0
@ 0x7f9e6c27961f clone at ??:0
W20260925 13:10:40.342519 17305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:10:40.342784 17305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:10:40.342836 17305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:10:40.344000 17305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:10:40.344064 17305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:10:40.344110 17305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:10:40.345589 17305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:40025
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=33403
--tserver_master_addrs=127.0.206.126:36617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:10:40.345855 17305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:10:40.346087 17305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:10:40.351228 17342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:40.351270 17341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:40.351420 17305 server_base.cc:1034] running on GCE node
W20260925 13:10:40.351326 17344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:40.351696 17305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:10:40.351958 17305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:10:40.353132 17305 hybrid_clock.cc:648] HybridClock initialized: now 1790341840353117 us; error 27 us; skew 500 ppm
Sep 25 13:10:40 dist-test-slave-chkf krb5kdc[16160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341840, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:10:40.356127 17305 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:10:40.356640 17305 webserver.cc:460] Webserver started at http://127.0.206.65:33403/ using document root <none> and password file <none>
I20260925 13:10:40.356882 17305 fs_manager.cc:359] Metadata directory not provided
I20260925 13:10:40.356942 17305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:10:40.358692 17305 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:10:40.359506 17351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:40.359719 17305 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:10:40.359819 17305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "2587f73a3c82462b8821e825761ef628"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "caa64392ce8701b60b5978a5cfd6569a"
server_key_iv: "498618e0fb6671293c960d3ff1f8be1b"
server_key_version: "encryptionkey@0"
I20260925 13:10:40.360273 17305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:10:40.374539 17305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:10:40.376927 17305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:10:40.377080 17305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:10:40.377311 17305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:10:40.377797 17358 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 13:10:40.379357 17305 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 13:10:40.379419 17305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 13:10:40.379454 17305 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 13:10:40.380332 17305 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 13:10:40.380383 17305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 13:10:40.380451 17358 tablet_bootstrap.cc:492] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628: Bootstrap starting.
I20260925 13:10:40.387399 17305 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:40025
I20260925 13:10:40.387537 17465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:40025 every 8 connection(s)
I20260925 13:10:40.387908 17305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:10:40.391088 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 17305
I20260925 13:10:40.391283 17358 log.cc:824] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628: Log is configured to *not* fsync() on all Append() calls
I20260925 13:10:40.391425 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:38717
--local_ip_for_outbound_sockets=127.0.206.66
--tserver_master_addrs=127.0.206.126:36617
--webserver_port=34287
--webserver_interface=127.0.206.66
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:10:40.393842 17358 tablet_bootstrap.cc:492] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 13:10:40.394155 17358 tablet_bootstrap.cc:492] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628: Bootstrap complete.
I20260925 13:10:40.395365 17358 ts_tablet_manager.cc:1397] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.003s
I20260925 13:10:40.397205 17358 raft_consensus.cc:348] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } }
I20260925 13:10:40.397614 17358 raft_consensus.cc:729] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2587f73a3c82462b8821e825761ef628, State: Initialized, Role: FOLLOWER
I20260925 13:10:40.397814 17358 consensus_queue.cc:260] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } }
I20260925 13:10:40.398110 17358 ts_tablet_manager.cc:1428] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
W20260925 13:10:40.461344 17468 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:10:40.388645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:53795)
0925 13:10:40.388861 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:40.388923 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:40.389665 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:40.461145 (+ 71480us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":95,"threads_started":1}
W20260925 13:10:40.461630 17466 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:36617 (0 consecutive failures): Network error: Failed to ping master at 127.0.206.126:36617: Client connection negotiation failed: client connection to 127.0.206.126:36617: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 13:10:40.462004 17468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:40.461879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:52611)
0925 13:10:40.461938 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:40.461981 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 13:10:40.462360 17468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:40.462260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:56937)
0925 13:10:40.462311 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:40.462339 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 13:10:40.462625 17466 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 13:10:40.474356 17470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:10:40.474558 17470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:10:40.474582 17470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:10:40.475754 17470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:10:40.475797 17470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:10:40.475824 17470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:10:40.477268 17470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:38717
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=34287
--tserver_master_addrs=127.0.206.126:36617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:10:40.477519 17470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:10:40.477746 17470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:10:40.482690 17470 server_base.cc:1034] running on GCE node
W20260925 13:10:40.482676 17477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:40.482671 17476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:40.482677 17479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:40.483172 17470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:10:40.483412 17470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:10:40.484549 17470 hybrid_clock.cc:648] HybridClock initialized: now 1790341840484534 us; error 34 us; skew 500 ppm
Sep 25 13:10:40 dist-test-slave-chkf krb5kdc[16160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341840, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:10:40.486912 17470 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:10:40.487360 17470 webserver.cc:460] Webserver started at http://127.0.206.66:34287/ using document root <none> and password file <none>
I20260925 13:10:40.487612 17470 fs_manager.cc:359] Metadata directory not provided
I20260925 13:10:40.487681 17470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:10:40.489362 17470 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 13:10:40.489909 17486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:40.490079 17470 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:10:40.490167 17470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "7099cb9663264a61bacd4a1c83989c26"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "6946de8ff053c97ddf4535605427b6c7"
server_key_iv: "8cecc3d4ba7b9583a8754a51c4cf7859"
server_key_version: "encryptionkey@0"
I20260925 13:10:40.490502 17470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:10:40.506860 17470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:10:40.508817 17470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:10:40.508971 17470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:10:40.509207 17470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:10:40.509509 17470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:10:40.509562 17470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:40.509606 17470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:10:40.509634 17470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:40.515499 17470 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:38717
I20260925 13:10:40.515553 17599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:38717 every 8 connection(s)
I20260925 13:10:40.515868 17470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:10:40.516861 17602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:40.516499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:52007)
0925 13:10:40.516654 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:40.516725 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
I20260925 13:10:40.517307 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 17470
W20260925 13:10:40.517369 17600 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:36617 (0 consecutive failures): Network error: Failed to ping master at 127.0.206.126:36617: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
I20260925 13:10:40.517652 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:43321
--local_ip_for_outbound_sockets=127.0.206.67
--tserver_master_addrs=127.0.206.126:36617
--webserver_port=45995
--webserver_interface=127.0.206.67
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:10:40.517678 17602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:40.517565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:39211)
0925 13:10:40.517618 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:40.517652 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 13:10:40.518034 17602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:40.517938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:51303)
0925 13:10:40.517990 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:40.518015 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 13:10:40.518167 17600 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 13:10:40.597846 17603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:10:40.598042 17603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:10:40.598065 17603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:10:40.599165 17603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:10:40.599207 17603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:10:40.599233 17603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:10:40.600643 17603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:43321
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=45995
--tserver_master_addrs=127.0.206.126:36617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:10:40.600836 17603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:10:40.601018 17603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:10:40.606124 17609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:40.606067 17611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:40.606171 17603 server_base.cc:1034] running on GCE node
W20260925 13:10:40.606067 17608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:40.606498 17603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:10:40.606751 17603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:10:40.607913 17603 hybrid_clock.cc:648] HybridClock initialized: now 1790341840607903 us; error 27 us; skew 500 ppm
Sep 25 13:10:40 dist-test-slave-chkf krb5kdc[16160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341840, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:10:40.610396 17603 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:10:40.610930 17603 webserver.cc:460] Webserver started at http://127.0.206.67:45995/ using document root <none> and password file <none>
I20260925 13:10:40.611177 17603 fs_manager.cc:359] Metadata directory not provided
I20260925 13:10:40.611248 17603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:10:40.612612 17603 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:10:40.613164 17618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:40.613329 17603 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:10:40.613435 17603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "c7418852b93f40139f3b93e08840d0df"
format_stamp: "Formatted at 2026-09-25 13:09:58 on dist-test-slave-chkf"
server_key: "4c65fa7b538a7bddb868277065beb27d"
server_key_iv: "9bdb79ffcbabff4b94445dfaf3cb9c59"
server_key_version: "encryptionkey@0"
I20260925 13:10:40.613782 17603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:10:40.628859 17603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:10:40.631183 17603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:10:40.631341 17603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:10:40.631573 17603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:10:40.631866 17603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:10:40.631918 17603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:40.631959 17603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:10:40.631986 17603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:40.637763 17603 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:43321
I20260925 13:10:40.637804 17731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:43321 every 8 connection(s)
I20260925 13:10:40.638170 17603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:10:40.639264 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:40.638898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:35573)
0925 13:10:40.639061 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:40.639129 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
W20260925 13:10:40.639662 17732 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:36617 (0 consecutive failures): Network error: Failed to ping master at 127.0.206.126:36617: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
I20260925 13:10:40.639928 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:40.639841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:47869)
0925 13:10:40.639882 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:40.639909 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 13:10:40.640211 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:40.640138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:46979)
0925 13:10:40.640175 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:40.640197 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260925 13:10:40.640331 17732 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 13:10:40.642585 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 17603
I20260925 13:10:41.463128 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:41.462910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:54947)
0925 13:10:41.463028 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:41.463066 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
I20260925 13:10:41.518743 17737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:41.518488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:39257)
0925 13:10:41.518618 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:41.518651 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260925 13:10:41.640830 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:41.640632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:52589)
0925 13:10:41.640751 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:41.640782 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
I20260925 13:10:42.234468 17739 raft_consensus.cc:482] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:10:42.234609 17739 raft_consensus.cc:504] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } }
I20260925 13:10:42.234982 17739 leader_election.cc:288] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7099cb9663264a61bacd4a1c83989c26 (127.0.206.66:38717), c7418852b93f40139f3b93e08840d0df (127.0.206.67:43321)
Sep 25 13:10:42 dist-test-slave-chkf krb5kdc[16160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341840, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.67@KRBTEST.COM
Sep 25 13:10:42 dist-test-slave-chkf krb5kdc[16160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341840, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.66@KRBTEST.COM
I20260925 13:10:42.245396 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:42.235357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:48663 (local address 127.0.206.67:43321)
0925 13:10:42.235492 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:42.235500 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:42.235517 (+ 17us) server_negotiation.cc:407] Connection header received
0925 13:10:42.235919 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:42.235921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:42.235961 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:42.236133 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:42.236895 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:42.237890 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:42.238963 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:42.239369 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:42.239485 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:42.241793 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:42.241797 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:42.241797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:10:42.241809 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:42.243685 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:42.244202 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:42.244205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:42.244205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:42.244315 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:42.244543 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:42.244544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:42.244544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:42.244825 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:42.244913 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:42.245160 (+ 247us) server_negotiation.cc:299] Negotiation successful
0925 13:10:42.245279 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":51,"threads_started":1}
I20260925 13:10:42.245383 17741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:42.235171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:43321 (local address 127.0.206.65:48663)
0925 13:10:42.235302 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:42.235314 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:42.235370 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:42.236164 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:42.236167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:42.236562 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:10:42.236764 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:42.236770 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:42.237990 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:42.237991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:42.238875 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:42.238878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:42.239463 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:42.239464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:42.239580 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:42.240174 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:10:42.240182 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:10:42.241686 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:10:42.243767 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:42.243774 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:42.243776 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:42.244103 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:42.244390 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:42.244390 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:42.244391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:42.244451 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:42.244909 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:10:42.244910 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:10:42.245006 (+ 96us) client_negotiation.cc:769] Sending connection context
0925 13:10:42.245103 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 13:10:42.245285 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":50,"thread_start_us":46,"threads_started":1}
I20260925 13:10:42.245422 17742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:42.235279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:48921 (local address 127.0.206.66:38717)
0925 13:10:42.235469 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:42.235480 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:42.235499 (+ 19us) server_negotiation.cc:407] Connection header received
0925 13:10:42.235919 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:42.235921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:42.235961 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:42.236133 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:42.236968 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:42.237902 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:42.238969 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:42.239366 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:42.239483 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:42.242172 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:42.242181 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:42.242182 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:10:42.242192 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:42.243799 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:42.244172 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:42.244175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:42.244176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:42.244268 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:42.244510 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:42.244511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:42.244512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:42.244783 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:42.244872 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:42.245162 (+ 290us) server_negotiation.cc:299] Negotiation successful
0925 13:10:42.245310 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":56,"threads_started":1}
I20260925 13:10:42.245496 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:42.235154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38717 (local address 127.0.206.65:48921)
0925 13:10:42.235275 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:42.235288 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:42.235350 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:42.236164 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:42.236167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:42.236564 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:10:42.236764 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:42.236770 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:42.238018 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:42.238019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:42.238873 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:42.238875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:42.239467 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:42.239468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:42.239582 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:42.240226 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:10:42.240231 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:10:42.242068 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:10:42.243881 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:42.243881 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:42.243882 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:42.244101 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:42.244333 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:42.244334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:42.244335 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:42.244449 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:42.244866 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:10:42.244872 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:10:42.244998 (+ 126us) client_negotiation.cc:769] Sending connection context
0925 13:10:42.245104 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 13:10:42.245362 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":180,"thread_start_us":76,"threads_started":1}
W20260925 13:10:42.246155 17353 leader_election.cc:341] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7099cb9663264a61bacd4a1c83989c26 (127.0.206.66:38717): Not found: Tablet not found: 596188104c974cd0a7e35fc9fc5bf5d9
W20260925 13:10:42.246817 17352 leader_election.cc:341] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c7418852b93f40139f3b93e08840d0df (127.0.206.67:43321): Not found: Tablet not found: 596188104c974cd0a7e35fc9fc5bf5d9
I20260925 13:10:42.246896 17352 leader_election.cc:302] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2587f73a3c82462b8821e825761ef628; no voters: 7099cb9663264a61bacd4a1c83989c26, c7418852b93f40139f3b93e08840d0df
I20260925 13:10:42.247004 17739 raft_consensus.cc:2726] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:10:42.463840 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:42.463620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:43603)
0925 13:10:42.463711 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:42.463793 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260925 13:10:42.519371 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:42.519161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:54311)
0925 13:10:42.519278 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:42.519328 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":55,"threads_started":1}
I20260925 13:10:42.641548 17745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:42.641313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:54919)
0925 13:10:42.641444 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:42.641493 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260925 13:10:43.464442 17746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:43.464254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:56823)
0925 13:10:43.464367 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:43.464398 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":58,"threads_started":1}
I20260925 13:10:43.520013 17747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:43.519837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:60485)
0925 13:10:43.519942 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:43.519975 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
I20260925 13:10:43.642186 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:43.641988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:47539)
0925 13:10:43.642108 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:43.642145 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
I20260925 13:10:44.269196 17749 raft_consensus.cc:482] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:10:44.269285 17749 raft_consensus.cc:504] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } }
I20260925 13:10:44.269452 17749 leader_election.cc:288] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7099cb9663264a61bacd4a1c83989c26 (127.0.206.66:38717), c7418852b93f40139f3b93e08840d0df (127.0.206.67:43321)
W20260925 13:10:44.269832 17353 leader_election.cc:341] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7099cb9663264a61bacd4a1c83989c26 (127.0.206.66:38717): Not found: Tablet not found: 596188104c974cd0a7e35fc9fc5bf5d9
W20260925 13:10:44.269920 17352 leader_election.cc:341] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c7418852b93f40139f3b93e08840d0df (127.0.206.67:43321): Not found: Tablet not found: 596188104c974cd0a7e35fc9fc5bf5d9
I20260925 13:10:44.270017 17352 leader_election.cc:302] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2587f73a3c82462b8821e825761ef628; no voters: 7099cb9663264a61bacd4a1c83989c26, c7418852b93f40139f3b93e08840d0df
I20260925 13:10:44.270105 17749 raft_consensus.cc:2726] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:10:44.465037 17750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:44.464831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:55915)
0925 13:10:44.464943 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:44.464976 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":56,"threads_started":1}
I20260925 13:10:44.520655 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:44.520421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:45035)
0925 13:10:44.520567 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:44.520602 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260925 13:10:44.642814 17752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:44.642612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:41537)
0925 13:10:44.642733 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:44.642765 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":54,"threads_started":1}
I20260925 13:10:45.465682 17753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:45.465468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:41889)
0925 13:10:45.465603 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:45.465636 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
I20260925 13:10:45.521313 17754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:45.521054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:58847)
0925 13:10:45.521241 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:45.521276 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":52,"threads_started":1}
I20260925 13:10:45.643381 17755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:45.643192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:59795)
0925 13:10:45.643297 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:45.643344 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
I20260925 13:10:46.466307 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:46.466072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:53071)
0925 13:10:46.466221 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:46.466261 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260925 13:10:46.521946 17757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:46.521742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:35105)
0925 13:10:46.521862 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:46.521897 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":76,"threads_started":1}
I20260925 13:10:46.643989 17758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:46.643807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:43741)
0925 13:10:46.643923 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:46.643957 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260925 13:10:46.857932 17759 raft_consensus.cc:482] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:10:46.858022 17759 raft_consensus.cc:504] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } }
I20260925 13:10:46.858183 17759 leader_election.cc:288] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7099cb9663264a61bacd4a1c83989c26 (127.0.206.66:38717), c7418852b93f40139f3b93e08840d0df (127.0.206.67:43321)
W20260925 13:10:46.858629 17353 leader_election.cc:341] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7099cb9663264a61bacd4a1c83989c26 (127.0.206.66:38717): Not found: Tablet not found: 596188104c974cd0a7e35fc9fc5bf5d9
W20260925 13:10:46.858772 17352 leader_election.cc:341] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c7418852b93f40139f3b93e08840d0df (127.0.206.67:43321): Not found: Tablet not found: 596188104c974cd0a7e35fc9fc5bf5d9
I20260925 13:10:46.858819 17352 leader_election.cc:302] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2587f73a3c82462b8821e825761ef628; no voters: 7099cb9663264a61bacd4a1c83989c26, c7418852b93f40139f3b93e08840d0df
I20260925 13:10:46.858901 17759 raft_consensus.cc:2726] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:10:47.466991 17760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:47.466787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:55901)
0925 13:10:47.466912 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:47.466946 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260925 13:10:47.522601 17761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:47.522395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:38175)
0925 13:10:47.522528 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:47.522561 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260925 13:10:47.644590 17762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:47.644390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:51507)
0925 13:10:47.644519 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:47.644552 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
I20260925 13:10:48.467732 17763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:48.467441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:57245)
0925 13:10:48.467642 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:48.467686 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":99,"threads_started":1}
I20260925 13:10:48.523340 17764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:48.523142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:38721)
0925 13:10:48.523260 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:48.523297 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
I20260925 13:10:48.645267 17765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:48.645019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:59053)
0925 13:10:48.645167 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:48.645204 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":57,"threads_started":1}
I20260925 13:10:49.468433 17766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:49.468233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:45435)
0925 13:10:49.468355 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:49.468391 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260925 13:10:49.523960 17767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:49.523774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:54011)
0925 13:10:49.523886 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:49.523920 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260925 13:10:49.645884 17768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:49.645690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:39183)
0925 13:10:49.645806 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:49.645843 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":57,"threads_started":1}
I20260925 13:10:49.691463 17769 raft_consensus.cc:482] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:10:49.691570 17769 raft_consensus.cc:504] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } }
I20260925 13:10:49.691763 17769 leader_election.cc:288] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7099cb9663264a61bacd4a1c83989c26 (127.0.206.66:38717), c7418852b93f40139f3b93e08840d0df (127.0.206.67:43321)
W20260925 13:10:49.692157 17352 leader_election.cc:341] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c7418852b93f40139f3b93e08840d0df (127.0.206.67:43321): Not found: Tablet not found: 596188104c974cd0a7e35fc9fc5bf5d9
W20260925 13:10:49.692236 17353 leader_election.cc:341] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7099cb9663264a61bacd4a1c83989c26 (127.0.206.66:38717): Not found: Tablet not found: 596188104c974cd0a7e35fc9fc5bf5d9
I20260925 13:10:49.692294 17353 leader_election.cc:302] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2587f73a3c82462b8821e825761ef628; no voters: 7099cb9663264a61bacd4a1c83989c26, c7418852b93f40139f3b93e08840d0df
I20260925 13:10:49.692436 17769 raft_consensus.cc:2726] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:10:50.469236 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:50.468997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:60047)
0925 13:10:50.469142 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:50.469185 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":61,"threads_started":1}
I20260925 13:10:50.524631 17771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:50.524425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:46831)
0925 13:10:50.524555 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:50.524588 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260925 13:10:50.646510 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:50.646320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:38145)
0925 13:10:50.646431 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:50.646469 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
I20260925 13:10:51.469975 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:51.469746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:54707)
0925 13:10:51.469892 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:51.469929 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260925 13:10:51.525446 17774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:51.525234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:35605)
0925 13:10:51.525365 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:51.525402 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
I20260925 13:10:51.647194 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:51.646966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:39283)
0925 13:10:51.647105 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:51.647151 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
I20260925 13:10:52.470795 17776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:52.470508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:49295)
0925 13:10:52.470652 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:52.470690 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
I20260925 13:10:52.526177 17777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:52.525910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:35599)
0925 13:10:52.526090 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:52.526131 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":83,"threads_started":1}
I20260925 13:10:52.647832 17778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:52.647646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:49851)
0925 13:10:52.647757 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:52.647791 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
I20260925 13:10:53.471421 17779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:53.471220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:48507)
0925 13:10:53.471338 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:53.471376 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260925 13:10:53.526883 17780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:53.526636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:56847)
0925 13:10:53.526801 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:53.526835 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":55,"threads_started":1}
I20260925 13:10:53.648447 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:53.648248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:52947)
0925 13:10:53.648362 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:53.648399 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260925 13:10:54.472060 17782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:54.471875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:47597)
0925 13:10:54.471985 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:54.472014 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260925 13:10:54.527555 17783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:54.527315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:47809)
0925 13:10:54.527457 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:54.527511 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
I20260925 13:10:54.649118 17784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:54.648899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:43861)
0925 13:10:54.649024 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:54.649065 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":55,"threads_started":1}
I20260925 13:10:55.098191 17785 raft_consensus.cc:482] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:10:55.098308 17785 raft_consensus.cc:504] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7099cb9663264a61bacd4a1c83989c26" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38717 } } peers { permanent_uuid: "c7418852b93f40139f3b93e08840d0df" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43321 } } peers { permanent_uuid: "2587f73a3c82462b8821e825761ef628" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 40025 } }
I20260925 13:10:55.098521 17785 leader_election.cc:288] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7099cb9663264a61bacd4a1c83989c26 (127.0.206.66:38717), c7418852b93f40139f3b93e08840d0df (127.0.206.67:43321)
W20260925 13:10:55.099016 17353 leader_election.cc:341] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7099cb9663264a61bacd4a1c83989c26 (127.0.206.66:38717): Not found: Tablet not found: 596188104c974cd0a7e35fc9fc5bf5d9
W20260925 13:10:55.099103 17352 leader_election.cc:341] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c7418852b93f40139f3b93e08840d0df (127.0.206.67:43321): Not found: Tablet not found: 596188104c974cd0a7e35fc9fc5bf5d9
I20260925 13:10:55.099167 17352 leader_election.cc:302] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2587f73a3c82462b8821e825761ef628; no voters: 7099cb9663264a61bacd4a1c83989c26, c7418852b93f40139f3b93e08840d0df
I20260925 13:10:55.099255 17785 raft_consensus.cc:2726] T 596188104c974cd0a7e35fc9fc5bf5d9 P 2587f73a3c82462b8821e825761ef628 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 13:10:55.472815 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:55.472589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.65:46423)
0925 13:10:55.472734 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:55.472768 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260925 13:10:55.528276 17787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:55.528050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.66:34219)
0925 13:10:55.528197 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:55.528233 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":77,"threads_started":1}
I20260925 13:10:55.649827 17788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:55.649630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36617 (local address 127.0.206.67:45171)
0925 13:10:55.649746 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:55.649784 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260925 13:10:55.653187 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 17305
I20260925 13:10:55.657770 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 17470
I20260925 13:10:55.662166 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 17603
2026-09-25T13:10:55Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58697 ms)
[----------] 30 tests from SecurityITest (228419 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:10:55 dist-test-slave-chkf krb5kdc[17791](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:10:55 dist-test-slave-chkf krb5kdc[17791](info): set up 2 sockets
Sep 25 13:10:55 dist-test-slave-chkf krb5kdc[17791](info): commencing operation
krb5kdc: starting...
W20260925 13:10:56.698290 825 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:10:56 dist-test-slave-chkf krb5kdc[17791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:10:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:10:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:10:56.810425 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:39471
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:39471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:10:56.890897 17804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:10:56.891117 17804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:10:56.891155 17804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:10:56.892267 17804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:10:56.892333 17804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:10:56.892357 17804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:10:56.892379 17804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:10:56.892407 17804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:10:56.893826 17804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:39471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:39471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:10:56.894096 17804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:10:56.894313 17804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:10:56.899271 17812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:56.899315 17810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:56.899331 17809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:56.899367 17804 server_base.cc:1034] running on GCE node
I20260925 13:10:56.899819 17804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:10:56.900104 17804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:10:56.901263 17804 hybrid_clock.cc:648] HybridClock initialized: now 1790341856901252 us; error 27 us; skew 500 ppm
Sep 25 13:10:56 dist-test-slave-chkf krb5kdc[17791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341856, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:10:56.903868 17804 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:10:56.904345 17804 webserver.cc:460] Webserver started at http://127.0.206.126:33655/ using document root <none> and password file <none>
I20260925 13:10:56.904580 17804 fs_manager.cc:359] Metadata directory not provided
I20260925 13:10:56.904646 17804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:10:56.904796 17804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:10:56.905822 17804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "35d31b5d2e0f41559c372340ab385365"
format_stamp: "Formatted at 2026-09-25 13:10:56 on dist-test-slave-chkf"
server_key: "24fd0d902272827a52c8f19a96f283ec"
server_key_iv: "5484ea1d009aa809739cb5616fbc87b4"
server_key_version: "encryptionkey@0"
I20260925 13:10:56.906189 17804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "35d31b5d2e0f41559c372340ab385365"
format_stamp: "Formatted at 2026-09-25 13:10:56 on dist-test-slave-chkf"
server_key: "24fd0d902272827a52c8f19a96f283ec"
server_key_iv: "5484ea1d009aa809739cb5616fbc87b4"
server_key_version: "encryptionkey@0"
I20260925 13:10:56.907544 17804 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:10:56.908217 17819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:56.908381 17804 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:10:56.908479 17804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "35d31b5d2e0f41559c372340ab385365"
format_stamp: "Formatted at 2026-09-25 13:10:56 on dist-test-slave-chkf"
server_key: "24fd0d902272827a52c8f19a96f283ec"
server_key_iv: "5484ea1d009aa809739cb5616fbc87b4"
server_key_version: "encryptionkey@0"
I20260925 13:10:56.908561 17804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:10:56.920063 17804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:10:56.922325 17804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:10:56.922487 17804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:10:56.926299 17804 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:39471
I20260925 13:10:56.926330 17871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:39471 every 8 connection(s)
I20260925 13:10:56.926666 17804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:10:56.927536 17872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:10:56.929600 17872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365: Bootstrap starting.
I20260925 13:10:56.930104 17872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365: Neither blocks nor log segments found. Creating new log.
I20260925 13:10:56.930322 17872 log.cc:824] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365: Log is configured to *not* fsync() on all Append() calls
I20260925 13:10:56.931030 17872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365: No bootstrap required, opened a new log
I20260925 13:10:56.932487 17872 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35d31b5d2e0f41559c372340ab385365" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39471 } }
I20260925 13:10:56.932616 17872 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:10:56.932636 17872 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35d31b5d2e0f41559c372340ab385365, State: Initialized, Role: FOLLOWER
I20260925 13:10:56.932762 17872 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35d31b5d2e0f41559c372340ab385365" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39471 } }
I20260925 13:10:56.932839 17872 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:10:56.932860 17872 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:10:56.932888 17872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:10:56.933342 17872 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35d31b5d2e0f41559c372340ab385365" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39471 } }
I20260925 13:10:56.933421 17872 leader_election.cc:302] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35d31b5d2e0f41559c372340ab385365; no voters:
I20260925 13:10:56.933537 17872 leader_election.cc:288] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:10:56.933621 17875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:10:56.933792 17875 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365 [term 1 LEADER]: Becoming Leader. State: Replica: 35d31b5d2e0f41559c372340ab385365, State: Running, Role: LEADER
I20260925 13:10:56.933836 17872 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:10:56.933916 17875 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35d31b5d2e0f41559c372340ab385365" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39471 } }
I20260925 13:10:56.934288 17877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35d31b5d2e0f41559c372340ab385365. Latest consensus state: current_term: 1 leader_uuid: "35d31b5d2e0f41559c372340ab385365" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35d31b5d2e0f41559c372340ab385365" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39471 } } }
I20260925 13:10:56.934410 17876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35d31b5d2e0f41559c372340ab385365" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35d31b5d2e0f41559c372340ab385365" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39471 } } }
I20260925 13:10:56.934492 17876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365 [sys.catalog]: This master's current role is: LEADER
I20260925 13:10:56.934661 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 17804
I20260925 13:10:56.934932 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:10:56.934438 17877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365 [sys.catalog]: This master's current role is: LEADER
I20260925 13:10:56.935164 825 external_mini_cluster.cc:1428] Setting key 0ed727ba0858a85078e2dbb0bcd8a9c6
W20260925 13:10:56.935734 17891 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:10:56.935786 17891 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:10:56.935880 17884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:10:56.936053 17884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:10:56.937655 17884 catalog_manager.cc:1347] Generated new cluster ID: 82febd13c9394ea2a7579a3824cc9156
I20260925 13:10:56.937708 17884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 13:10:56 dist-test-slave-chkf krb5kdc[17791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:10:56.945011 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:56.935625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56750 (local address 127.0.206.126:39471)
0925 13:10:56.935798 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:56.935808 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:56.936127 (+ 319us) server_negotiation.cc:407] Connection header received
0925 13:10:56.936241 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:56.936247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:56.936300 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:56.936430 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:56.937170 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:56.938266 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:56.939124 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:56.939540 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:56.939677 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:56.941949 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:56.941959 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:56.941960 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:10:56.941970 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:56.943559 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:56.943915 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:56.943918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:56.943918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:56.944013 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:56.944235 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:56.944236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:56.944236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:56.944447 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:56.944600 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:56.944705 (+ 105us) server_negotiation.cc:299] Negotiation successful
0925 13:10:56.944837 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
I20260925 13:10:56.955193 17884 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:10:56.955888 17884 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:10:56.964066 17884 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365: Generated new TSK 0
I20260925 13:10:56.964236 17884 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260925 13:10:56.997423 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39471
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:10:57.077414 17898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:10:57.077613 17898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:10:57.077636 17898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:10:57.078788 17898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:10:57.078830 17898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:10:57.078857 17898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:10:57.080364 17898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:10:57.080545 17898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:10:57.080711 17898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:10:57.085657 17903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:57.085654 17904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:57.085657 17906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:57.085987 17898 server_base.cc:1034] running on GCE node
I20260925 13:10:57.086211 17898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:10:57.086442 17898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:10:57.087604 17898 hybrid_clock.cc:648] HybridClock initialized: now 1790341857087595 us; error 39 us; skew 500 ppm
Sep 25 13:10:57 dist-test-slave-chkf krb5kdc[17791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341857, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:10:57.090077 17898 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:10:57.090481 17898 webserver.cc:460] Webserver started at http://127.0.206.65:34615/ using document root <none> and password file <none>
I20260925 13:10:57.090775 17898 fs_manager.cc:359] Metadata directory not provided
I20260925 13:10:57.090850 17898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:10:57.090986 17898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:10:57.092006 17898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "a390ef3f9ebc4a89a76d72987eea1517"
format_stamp: "Formatted at 2026-09-25 13:10:57 on dist-test-slave-chkf"
server_key: "0772b9491fa44952501b3eb62671ce56"
server_key_iv: "9c20bb79ea2f717203928515d88caa81"
server_key_version: "encryptionkey@0"
I20260925 13:10:57.092356 17898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "a390ef3f9ebc4a89a76d72987eea1517"
format_stamp: "Formatted at 2026-09-25 13:10:57 on dist-test-slave-chkf"
server_key: "0772b9491fa44952501b3eb62671ce56"
server_key_iv: "9c20bb79ea2f717203928515d88caa81"
server_key_version: "encryptionkey@0"
I20260925 13:10:57.093637 17898 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:10:57.094353 17913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:57.094530 17898 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:10:57.094628 17898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "a390ef3f9ebc4a89a76d72987eea1517"
format_stamp: "Formatted at 2026-09-25 13:10:57 on dist-test-slave-chkf"
server_key: "0772b9491fa44952501b3eb62671ce56"
server_key_iv: "9c20bb79ea2f717203928515d88caa81"
server_key_version: "encryptionkey@0"
I20260925 13:10:57.094733 17898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:10:57.108418 17898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:10:57.110356 17898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:10:57.110515 17898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:10:57.110803 17898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:10:57.111156 17898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:10:57.111197 17898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:57.111236 17898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:10:57.111263 17898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:57.117169 17898 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:32973
I20260925 13:10:57.117211 18026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:32973 every 8 connection(s)
I20260925 13:10:57.117517 17898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:10:57.122124 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 17898
I20260925 13:10:57.122246 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:10:57.122501 825 external_mini_cluster.cc:1428] Setting key 2d589363358e63787a31149c0c5be47c
Sep 25 13:10:57 dist-test-slave-chkf krb5kdc[17791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341857, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:10:57.127724 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.118267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:46659 (local address 127.0.206.126:39471)
0925 13:10:57.118311 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:57.118313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:57.118563 (+ 250us) server_negotiation.cc:407] Connection header received
0925 13:10:57.119564 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:57.119565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:57.119581 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:57.119633 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:57.120239 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.120759 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.121761 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.121970 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.122054 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.124683 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:57.124684 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:57.124685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:10:57.124694 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:57.126209 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:57.126692 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:57.126693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:57.126693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:57.126802 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:57.127055 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:57.127057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:57.127057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:57.127174 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:57.127252 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:57.127524 (+ 272us) server_negotiation.cc:299] Negotiation successful
0925 13:10:57.127635 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 13:10:57.127887 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.118246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:39471 (local address 127.0.206.65:46659)
0925 13:10:57.118431 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:57.118465 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:57.119439 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:57.119700 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:57.119704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:57.119960 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:10:57.120144 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:57.120148 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.120854 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.120855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:57.121670 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:57.121672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.122071 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.122072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:57.122193 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.122822 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:10:57.122832 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:10:57.124581 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:10:57.126312 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:57.126320 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:57.126323 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:57.126546 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:57.126870 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:57.126871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:57.126872 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:57.126986 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:57.127244 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:10:57.127250 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:10:57.127390 (+ 140us) client_negotiation.cc:769] Sending connection context
0925 13:10:57.127502 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 13:10:57.127657 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260925 13:10:57.128322 18027 heartbeater.cc:347] Connected to a master server at 127.0.206.126:39471
I20260925 13:10:57.128412 18027 heartbeater.cc:464] Registering TS with master...
I20260925 13:10:57.128607 18027 heartbeater.cc:511] Master 127.0.206.126:39471 requested a full tablet report, sending...
I20260925 13:10:57.129119 17836 ts_manager.cc:194] Registered new tserver with Master: a390ef3f9ebc4a89a76d72987eea1517 (127.0.206.65:32973)
I20260925 13:10:57.129733 17836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:46659
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260925 13:10:57.147233 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39471
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:10:57.226610 18032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:10:57.226815 18032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:10:57.226837 18032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:10:57.227998 18032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:10:57.228039 18032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:10:57.228091 18032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:10:57.229514 18032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:10:57.229760 18032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:10:57.229983 18032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:10:57.234965 18040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:57.235016 18038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:57.235073 18037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:57.235025 18032 server_base.cc:1034] running on GCE node
I20260925 13:10:57.235287 18032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:10:57.235524 18032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:10:57.236706 18032 hybrid_clock.cc:648] HybridClock initialized: now 1790341857236693 us; error 29 us; skew 500 ppm
Sep 25 13:10:57 dist-test-slave-chkf krb5kdc[17791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341857, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:10:57.239231 18032 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:10:57.239669 18032 webserver.cc:460] Webserver started at http://127.0.206.66:41993/ using document root <none> and password file <none>
I20260925 13:10:57.239888 18032 fs_manager.cc:359] Metadata directory not provided
I20260925 13:10:57.239933 18032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:10:57.240085 18032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:10:57.241110 18032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "030ce18d31ea424492d822fc33b185ae"
format_stamp: "Formatted at 2026-09-25 13:10:57 on dist-test-slave-chkf"
server_key: "96babc62f1c81df31ae9d51661567b8b"
server_key_iv: "82a57b0bd45ec828ff5e88c9fc20ed5c"
server_key_version: "encryptionkey@0"
I20260925 13:10:57.241477 18032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "030ce18d31ea424492d822fc33b185ae"
format_stamp: "Formatted at 2026-09-25 13:10:57 on dist-test-slave-chkf"
server_key: "96babc62f1c81df31ae9d51661567b8b"
server_key_iv: "82a57b0bd45ec828ff5e88c9fc20ed5c"
server_key_version: "encryptionkey@0"
I20260925 13:10:57.242930 18032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 13:10:57.243683 18047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:57.244024 18032 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:10:57.244129 18032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "030ce18d31ea424492d822fc33b185ae"
format_stamp: "Formatted at 2026-09-25 13:10:57 on dist-test-slave-chkf"
server_key: "96babc62f1c81df31ae9d51661567b8b"
server_key_iv: "82a57b0bd45ec828ff5e88c9fc20ed5c"
server_key_version: "encryptionkey@0"
I20260925 13:10:57.244199 18032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:10:57.257566 18032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:10:57.259477 18032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:10:57.259631 18032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:10:57.259855 18032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:10:57.260142 18032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:10:57.260174 18032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:57.260196 18032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:10:57.260241 18032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:57.266005 18032 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:44501
I20260925 13:10:57.266047 18160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:44501 every 8 connection(s)
I20260925 13:10:57.266348 18032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:10:57.271888 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 18032
I20260925 13:10:57.272017 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:10:57.272241 825 external_mini_cluster.cc:1428] Setting key bc909648dbe237d930c3ff3c4b7c51a1
Sep 25 13:10:57 dist-test-slave-chkf krb5kdc[17791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341857, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:10:57.275975 18163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.267004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:39471 (local address 127.0.206.66:40367)
0925 13:10:57.267179 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:57.267219 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:57.268142 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:57.268389 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:57.268393 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:57.268653 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:10:57.268835 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:57.268839 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.269559 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.269561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:57.270357 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:57.270359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.270768 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.270769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:57.270897 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.271402 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:10:57.271410 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:10:57.273059 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:10:57.274570 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:57.274576 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:57.274578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:57.274866 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:57.275103 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:57.275103 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:57.275104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:57.275211 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:57.275510 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:10:57.275515 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:10:57.275626 (+ 111us) client_negotiation.cc:769] Sending connection context
0925 13:10:57.275715 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 13:10:57.275827 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260925 13:10:57.275995 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.267038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:40367 (local address 127.0.206.126:39471)
0925 13:10:57.267087 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:57.267089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:57.267308 (+ 219us) server_negotiation.cc:407] Connection header received
0925 13:10:57.268255 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:57.268256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:57.268268 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:57.268322 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:57.268936 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.269449 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.270454 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.270666 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.270763 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.273154 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:57.273155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:57.273156 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:10:57.273163 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:57.274471 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:57.274956 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:57.274958 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:57.274958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:57.275020 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:57.275290 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:57.275291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:57.275291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:57.275428 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:57.275496 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:57.275759 (+ 263us) server_negotiation.cc:299] Negotiation successful
0925 13:10:57.275855 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 13:10:57.276420 18161 heartbeater.cc:347] Connected to a master server at 127.0.206.126:39471
I20260925 13:10:57.276516 18161 heartbeater.cc:464] Registering TS with master...
I20260925 13:10:57.276698 18161 heartbeater.cc:511] Master 127.0.206.126:39471 requested a full tablet report, sending...
I20260925 13:10:57.277099 17836 ts_manager.cc:194] Registered new tserver with Master: 030ce18d31ea424492d822fc33b185ae (127.0.206.66:44501)
I20260925 13:10:57.277459 17836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:40367
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260925 13:10:57.298137 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39471
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:10:57.378087 18166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:10:57.378310 18166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:10:57.378350 18166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:10:57.379585 18166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:10:57.379633 18166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:10:57.379673 18166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:10:57.381758 18166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:10:57.381937 18166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:10:57.382174 18166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:10:57.387118 18166 server_base.cc:1034] running on GCE node
W20260925 13:10:57.387022 18171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:57.387039 18174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:57.387061 18172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:57.387418 18166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:10:57.387689 18166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:10:57.388854 18166 hybrid_clock.cc:648] HybridClock initialized: now 1790341857388831 us; error 34 us; skew 500 ppm
Sep 25 13:10:57 dist-test-slave-chkf krb5kdc[17791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341857, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:10:57.391327 18166 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:10:57.391767 18166 webserver.cc:460] Webserver started at http://127.0.206.67:44101/ using document root <none> and password file <none>
I20260925 13:10:57.391942 18166 fs_manager.cc:359] Metadata directory not provided
I20260925 13:10:57.391981 18166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:10:57.392067 18166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:10:57.393083 18166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "da5d928a2c9c4399baa9edb532cc6214"
format_stamp: "Formatted at 2026-09-25 13:10:57 on dist-test-slave-chkf"
server_key: "996b9b3976b218e2bf42d2af53fbbacb"
server_key_iv: "253313574f796b348f7ea68afdcd8773"
server_key_version: "encryptionkey@0"
I20260925 13:10:57.393380 18166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "da5d928a2c9c4399baa9edb532cc6214"
format_stamp: "Formatted at 2026-09-25 13:10:57 on dist-test-slave-chkf"
server_key: "996b9b3976b218e2bf42d2af53fbbacb"
server_key_iv: "253313574f796b348f7ea68afdcd8773"
server_key_version: "encryptionkey@0"
I20260925 13:10:57.394622 18166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:10:57.395407 18181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:57.395591 18166 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:10:57.395680 18166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "da5d928a2c9c4399baa9edb532cc6214"
format_stamp: "Formatted at 2026-09-25 13:10:57 on dist-test-slave-chkf"
server_key: "996b9b3976b218e2bf42d2af53fbbacb"
server_key_iv: "253313574f796b348f7ea68afdcd8773"
server_key_version: "encryptionkey@0"
I20260925 13:10:57.395753 18166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:10:57.423501 18166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:10:57.425396 18166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:10:57.425575 18166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:10:57.425827 18166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:10:57.426138 18166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:10:57.426172 18166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:57.426192 18166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:10:57.426237 18166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:57.432832 18166 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:40383
I20260925 13:10:57.432909 18294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:40383 every 8 connection(s)
I20260925 13:10:57.433213 18166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
Sep 25 13:10:57 dist-test-slave-chkf krb5kdc[17791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341857, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:10:57.442902 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 18166
I20260925 13:10:57.443017 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:10:57.443234 825 external_mini_cluster.cc:1428] Setting key b341b1135c9832c89568f88579d190e1
I20260925 13:10:57.444645 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.434291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:43431 (local address 127.0.206.126:39471)
0925 13:10:57.434334 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:57.434335 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:57.434365 (+ 30us) server_negotiation.cc:407] Connection header received
0925 13:10:57.435467 (+ 1102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:57.435468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:57.435480 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:57.435536 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:57.436177 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.436709 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.438338 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.438718 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.438783 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.441497 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:57.441498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:57.441498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:10:57.441506 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:57.443259 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:57.443766 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:57.443767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:57.443767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:57.443825 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:57.444130 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:57.444131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:57.444131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:57.444268 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:57.444326 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:57.444526 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 13:10:57.444583 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 13:10:57.444866 18297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.434029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:39471 (local address 127.0.206.67:43431)
0925 13:10:57.434211 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:57.434291 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:57.435363 (+ 1072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:57.435600 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:57.435604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:57.435855 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:10:57.436041 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:57.436045 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.436805 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.436806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:57.438238 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:57.438242 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.438796 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.438797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:57.438961 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.439608 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:10:57.439620 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:10:57.441391 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:10:57.443357 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:57.443363 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:57.443366 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:57.443698 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:57.443884 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:57.443885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:57.443885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:57.444023 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:57.444319 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:10:57.444323 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:10:57.444434 (+ 111us) client_negotiation.cc:769] Sending connection context
0925 13:10:57.444514 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 13:10:57.444636 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":70,"threads_started":1}
I20260925 13:10:57.445350 18295 heartbeater.cc:347] Connected to a master server at 127.0.206.126:39471
I20260925 13:10:57.445428 18295 heartbeater.cc:464] Registering TS with master...
I20260925 13:10:57.445623 18295 heartbeater.cc:511] Master 127.0.206.126:39471 requested a full tablet report, sending...
I20260925 13:10:57.445974 17836 ts_manager.cc:194] Registered new tserver with Master: da5d928a2c9c4399baa9edb532cc6214 (127.0.206.67:40383)
I20260925 13:10:57.446331 17836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:43431
I20260925 13:10:57.455051 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:10:57.461580 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.456333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56766 (local address 127.0.206.126:39471)
0925 13:10:57.456372 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:57.456373 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:57.456470 (+ 97us) server_negotiation.cc:407] Connection header received
0925 13:10:57.456520 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:57.456520 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:57.456529 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:57.456581 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:57.457087 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.457551 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.458231 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.458403 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.458449 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.459552 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:57.459553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:57.459554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:10:57.459559 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:57.460701 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:57.461002 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:57.461003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:57.461003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:57.461049 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:57.461183 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:57.461183 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:57.461184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:57.461306 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:57.461380 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:57.461467 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 13:10:57.461522 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 13:10:57.463171 17836 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:10:57.464797 17836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:10:57.471042 18306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.467039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:44501 (local address 127.0.0.1:58780)
0925 13:10:57.467276 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:57.467286 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:57.467328 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:57.467604 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:57.467607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:57.467609 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:10:57.467673 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:57.467676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.468701 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.468702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:57.469888 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:57.469890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.470732 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.470733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:57.470812 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.470825 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 13:10:57.470896 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 13:10:57.470950 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":90,"threads_started":1}
I20260925 13:10:57.471251 18309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.467126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44234 (local address 127.0.206.65:32973)
0925 13:10:57.467482 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:57.467487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:57.467497 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:10:57.467537 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:57.467539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:57.467575 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:57.467656 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:10:57.467838 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.468909 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.470128 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.470820 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.470895 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.470947 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:57.471030 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 13:10:57.471071 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":73,"threads_started":1}
I20260925 13:10:57.471282 18311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.467275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58780 (local address 127.0.206.66:44501)
0925 13:10:57.467431 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:57.467436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:57.467448 (+ 12us) server_negotiation.cc:407] Connection header received
0925 13:10:57.467496 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:57.467497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:57.467509 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:57.467577 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:10:57.467761 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.468584 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.469984 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.470609 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.470721 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.470781 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:57.471164 (+ 383us) server_negotiation.cc:299] Negotiation successful
0925 13:10:57.471208 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":64,"threads_started":1}
I20260925 13:10:57.471488 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.467039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:40383 (local address 127.0.0.1:47640)
0925 13:10:57.467404 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:57.467413 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:57.467451 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:57.468019 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:57.468020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:57.468021 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:10:57.468062 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:57.468064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.469246 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.469247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:57.470258 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:57.470261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.471239 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.471240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:57.471308 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.471313 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 13:10:57.471370 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 13:10:57.471416 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":2,"thread_start_us":86,"threads_started":1}
I20260925 13:10:57.471724 18310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.467264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47640 (local address 127.0.206.67:40383)
0925 13:10:57.467807 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:57.467812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:57.467820 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:10:57.467856 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:57.467857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:57.467871 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:57.467939 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:10:57.468146 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.469146 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.470343 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.471083 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.471153 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.471205 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:57.471606 (+ 401us) server_negotiation.cc:299] Negotiation successful
0925 13:10:57.471646 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":78,"threads_started":1}
I20260925 13:10:57.472075 18095 tablet_service.cc:1467] Processing CreateTablet for tablet 20d28ffc0eb74671965ef80c1b36329e (DEFAULT_TABLE table=test-table [id=4b3747c5fc14499b991c07f9eae5bdbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:10:57.472414 18095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20d28ffc0eb74671965ef80c1b36329e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:10:57.472334 18229 tablet_service.cc:1467] Processing CreateTablet for tablet 20d28ffc0eb74671965ef80c1b36329e (DEFAULT_TABLE table=test-table [id=4b3747c5fc14499b991c07f9eae5bdbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:10:57.472659 18229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20d28ffc0eb74671965ef80c1b36329e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:10:57.474871 18307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.467014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:32973 (local address 127.0.0.1:44234)
0925 13:10:57.467195 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:57.467233 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:57.467302 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:57.467690 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:57.467692 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:57.467694 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:10:57.467750 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:57.467752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.469013 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.469014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:57.470043 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:57.470045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.470901 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.470902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:57.470954 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.470956 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:10:57.471016 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 13:10:57.471044 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":68,"threads_started":1}
I20260925 13:10:57.474860 18312 tablet_bootstrap.cc:492] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae: Bootstrap starting.
I20260925 13:10:57.475096 18313 tablet_bootstrap.cc:492] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214: Bootstrap starting.
I20260925 13:10:57.475621 17961 tablet_service.cc:1467] Processing CreateTablet for tablet 20d28ffc0eb74671965ef80c1b36329e (DEFAULT_TABLE table=test-table [id=4b3747c5fc14499b991c07f9eae5bdbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:10:57.475767 18313 tablet_bootstrap.cc:654] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214: Neither blocks nor log segments found. Creating new log.
I20260925 13:10:57.475809 18312 tablet_bootstrap.cc:654] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae: Neither blocks nor log segments found. Creating new log.
I20260925 13:10:57.475929 17961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20d28ffc0eb74671965ef80c1b36329e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:10:57.476398 18312 log.cc:824] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae: Log is configured to *not* fsync() on all Append() calls
I20260925 13:10:57.476398 18313 log.cc:824] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214: Log is configured to *not* fsync() on all Append() calls
I20260925 13:10:57.477159 18313 tablet_bootstrap.cc:492] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214: No bootstrap required, opened a new log
I20260925 13:10:57.477232 18313 ts_tablet_manager.cc:1397] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:10:57.477348 18312 tablet_bootstrap.cc:492] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae: No bootstrap required, opened a new log
I20260925 13:10:57.477407 18312 ts_tablet_manager.cc:1397] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 13:10:57.478757 18316 tablet_bootstrap.cc:492] T 20d28ffc0eb74671965ef80c1b36329e P a390ef3f9ebc4a89a76d72987eea1517: Bootstrap starting.
I20260925 13:10:57.479235 18313 raft_consensus.cc:348] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a390ef3f9ebc4a89a76d72987eea1517" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32973 } } peers { permanent_uuid: "da5d928a2c9c4399baa9edb532cc6214" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 40383 } } peers { permanent_uuid: "030ce18d31ea424492d822fc33b185ae" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44501 } }
I20260925 13:10:57.479380 18316 tablet_bootstrap.cc:654] T 20d28ffc0eb74671965ef80c1b36329e P a390ef3f9ebc4a89a76d72987eea1517: Neither blocks nor log segments found. Creating new log.
I20260925 13:10:57.479414 18313 raft_consensus.cc:374] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:10:57.479450 18313 raft_consensus.cc:729] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da5d928a2c9c4399baa9edb532cc6214, State: Initialized, Role: FOLLOWER
I20260925 13:10:57.479378 18312 raft_consensus.cc:348] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a390ef3f9ebc4a89a76d72987eea1517" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32973 } } peers { permanent_uuid: "da5d928a2c9c4399baa9edb532cc6214" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 40383 } } peers { permanent_uuid: "030ce18d31ea424492d822fc33b185ae" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44501 } }
I20260925 13:10:57.479529 18312 raft_consensus.cc:374] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:10:57.479561 18312 raft_consensus.cc:729] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 030ce18d31ea424492d822fc33b185ae, State: Initialized, Role: FOLLOWER
I20260925 13:10:57.479578 18313 consensus_queue.cc:260] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a390ef3f9ebc4a89a76d72987eea1517" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32973 } } peers { permanent_uuid: "da5d928a2c9c4399baa9edb532cc6214" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 40383 } } peers { permanent_uuid: "030ce18d31ea424492d822fc33b185ae" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44501 } }
I20260925 13:10:57.479684 18312 consensus_queue.cc:260] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a390ef3f9ebc4a89a76d72987eea1517" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32973 } } peers { permanent_uuid: "da5d928a2c9c4399baa9edb532cc6214" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 40383 } } peers { permanent_uuid: "030ce18d31ea424492d822fc33b185ae" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44501 } }
I20260925 13:10:57.479853 18313 ts_tablet_manager.cc:1428] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 13:10:57.479911 18295 heartbeater.cc:503] Master 127.0.206.126:39471 was elected leader, sending a full tablet report...
I20260925 13:10:57.480007 18312 ts_tablet_manager.cc:1428] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 13:10:57.480103 18316 log.cc:824] T 20d28ffc0eb74671965ef80c1b36329e P a390ef3f9ebc4a89a76d72987eea1517: Log is configured to *not* fsync() on all Append() calls
I20260925 13:10:57.480134 18161 heartbeater.cc:503] Master 127.0.206.126:39471 was elected leader, sending a full tablet report...
I20260925 13:10:57.480876 18316 tablet_bootstrap.cc:492] T 20d28ffc0eb74671965ef80c1b36329e P a390ef3f9ebc4a89a76d72987eea1517: No bootstrap required, opened a new log
I20260925 13:10:57.480944 18316 ts_tablet_manager.cc:1397] T 20d28ffc0eb74671965ef80c1b36329e P a390ef3f9ebc4a89a76d72987eea1517: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 13:10:57.482478 18316 raft_consensus.cc:348] T 20d28ffc0eb74671965ef80c1b36329e P a390ef3f9ebc4a89a76d72987eea1517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a390ef3f9ebc4a89a76d72987eea1517" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32973 } } peers { permanent_uuid: "da5d928a2c9c4399baa9edb532cc6214" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 40383 } } peers { permanent_uuid: "030ce18d31ea424492d822fc33b185ae" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44501 } }
I20260925 13:10:57.482615 18316 raft_consensus.cc:374] T 20d28ffc0eb74671965ef80c1b36329e P a390ef3f9ebc4a89a76d72987eea1517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:10:57.482637 18316 raft_consensus.cc:729] T 20d28ffc0eb74671965ef80c1b36329e P a390ef3f9ebc4a89a76d72987eea1517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a390ef3f9ebc4a89a76d72987eea1517, State: Initialized, Role: FOLLOWER
I20260925 13:10:57.482775 18316 consensus_queue.cc:260] T 20d28ffc0eb74671965ef80c1b36329e P a390ef3f9ebc4a89a76d72987eea1517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a390ef3f9ebc4a89a76d72987eea1517" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32973 } } peers { permanent_uuid: "da5d928a2c9c4399baa9edb532cc6214" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 40383 } } peers { permanent_uuid: "030ce18d31ea424492d822fc33b185ae" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44501 } }
I20260925 13:10:57.483012 18316 ts_tablet_manager.cc:1428] T 20d28ffc0eb74671965ef80c1b36329e P a390ef3f9ebc4a89a76d72987eea1517: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:10:57.483047 18027 heartbeater.cc:503] Master 127.0.206.126:39471 was elected leader, sending a full tablet report...
W20260925 13:10:57.516852 18162 tablet.cc:2369] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:10:57.618247 18028 tablet.cc:2369] T 20d28ffc0eb74671965ef80c1b36329e P a390ef3f9ebc4a89a76d72987eea1517: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:10:57.683784 18296 tablet.cc:2369] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:10:57.728480 18318 raft_consensus.cc:482] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:10:57.728608 18318 raft_consensus.cc:504] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a390ef3f9ebc4a89a76d72987eea1517" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32973 } } peers { permanent_uuid: "da5d928a2c9c4399baa9edb532cc6214" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 40383 } } peers { permanent_uuid: "030ce18d31ea424492d822fc33b185ae" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44501 } }
I20260925 13:10:57.729149 18318 leader_election.cc:288] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a390ef3f9ebc4a89a76d72987eea1517 (127.0.206.65:32973), da5d928a2c9c4399baa9edb532cc6214 (127.0.206.67:40383)
I20260925 13:10:57.733027 18163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.729344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:40383 (local address 127.0.206.66:36437)
0925 13:10:57.729411 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:57.729424 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:57.729473 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:57.729645 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:57.729646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:57.729648 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:10:57.729729 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:57.729731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.730686 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.730687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:57.731958 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:57.731961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.732708 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.732710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:57.732804 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.732811 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 13:10:57.732881 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 13:10:57.732932 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":60}
I20260925 13:10:57.733099 18309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.729443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:37731 (local address 127.0.206.65:32973)
0925 13:10:57.729498 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:57.729500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:57.729699 (+ 199us) server_negotiation.cc:407] Connection header received
0925 13:10:57.729741 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:57.729742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:57.729752 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:57.729823 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:10:57.729957 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.730679 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.732050 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.732605 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.732685 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.732693 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:57.732987 (+ 294us) server_negotiation.cc:299] Negotiation successful
0925 13:10:57.733030 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 13:10:57.733124 18310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.729435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:36437 (local address 127.0.206.67:40383)
0925 13:10:57.729501 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:57.729503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:57.729514 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:10:57.729545 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:57.729546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:57.729558 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:57.729628 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:10:57.729807 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.730609 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.732051 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.732605 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.732685 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.732693 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:57.732987 (+ 294us) server_negotiation.cc:299] Negotiation successful
0925 13:10:57.733033 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 13:10:57.733034 18321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.729345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:32973 (local address 127.0.206.66:37731)
0925 13:10:57.729593 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:57.729622 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:57.729681 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:57.729841 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:57.729842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:57.729843 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:10:57.729891 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:57.729892 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.730769 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.730770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:57.731958 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:57.731961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.732708 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.732710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:57.732804 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.732811 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 13:10:57.732881 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 13:10:57.732932 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":83,"thread_start_us":109,"threads_started":1}
I20260925 13:10:57.733459 18249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d28ffc0eb74671965ef80c1b36329e" candidate_uuid: "030ce18d31ea424492d822fc33b185ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da5d928a2c9c4399baa9edb532cc6214" is_pre_election: true
I20260925 13:10:57.733623 18249 raft_consensus.cc:2454] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 030ce18d31ea424492d822fc33b185ae in term 0.
I20260925 13:10:57.733695 17981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d28ffc0eb74671965ef80c1b36329e" candidate_uuid: "030ce18d31ea424492d822fc33b185ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a390ef3f9ebc4a89a76d72987eea1517" is_pre_election: true
I20260925 13:10:57.733836 17981 raft_consensus.cc:2454] T 20d28ffc0eb74671965ef80c1b36329e P a390ef3f9ebc4a89a76d72987eea1517 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 030ce18d31ea424492d822fc33b185ae in term 0.
I20260925 13:10:57.733863 18048 leader_election.cc:302] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 030ce18d31ea424492d822fc33b185ae, da5d928a2c9c4399baa9edb532cc6214; no voters:
I20260925 13:10:57.733978 18318 raft_consensus.cc:2781] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:10:57.734042 18318 raft_consensus.cc:482] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:10:57.734066 18318 raft_consensus.cc:3037] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:10:57.734886 18318 raft_consensus.cc:504] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a390ef3f9ebc4a89a76d72987eea1517" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32973 } } peers { permanent_uuid: "da5d928a2c9c4399baa9edb532cc6214" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 40383 } } peers { permanent_uuid: "030ce18d31ea424492d822fc33b185ae" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44501 } }
I20260925 13:10:57.735031 18318 leader_election.cc:288] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [CANDIDATE]: Term 1 election: Requested vote from peers a390ef3f9ebc4a89a76d72987eea1517 (127.0.206.65:32973), da5d928a2c9c4399baa9edb532cc6214 (127.0.206.67:40383)
I20260925 13:10:57.735173 17981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d28ffc0eb74671965ef80c1b36329e" candidate_uuid: "030ce18d31ea424492d822fc33b185ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a390ef3f9ebc4a89a76d72987eea1517"
I20260925 13:10:57.735272 17981 raft_consensus.cc:3037] T 20d28ffc0eb74671965ef80c1b36329e P a390ef3f9ebc4a89a76d72987eea1517 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:10:57.735962 17981 raft_consensus.cc:2454] T 20d28ffc0eb74671965ef80c1b36329e P a390ef3f9ebc4a89a76d72987eea1517 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 030ce18d31ea424492d822fc33b185ae in term 1.
I20260925 13:10:57.736090 18249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d28ffc0eb74671965ef80c1b36329e" candidate_uuid: "030ce18d31ea424492d822fc33b185ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da5d928a2c9c4399baa9edb532cc6214"
I20260925 13:10:57.736161 18050 leader_election.cc:302] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 030ce18d31ea424492d822fc33b185ae, a390ef3f9ebc4a89a76d72987eea1517; no voters:
I20260925 13:10:57.736181 18249 raft_consensus.cc:3037] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:10:57.736253 18318 raft_consensus.cc:2781] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:10:57.736430 18318 raft_consensus.cc:686] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [term 1 LEADER]: Becoming Leader. State: Replica: 030ce18d31ea424492d822fc33b185ae, State: Running, Role: LEADER
I20260925 13:10:57.736526 18318 consensus_queue.cc:237] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a390ef3f9ebc4a89a76d72987eea1517" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32973 } } peers { permanent_uuid: "da5d928a2c9c4399baa9edb532cc6214" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 40383 } } peers { permanent_uuid: "030ce18d31ea424492d822fc33b185ae" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44501 } }
I20260925 13:10:57.737061 18249 raft_consensus.cc:2454] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 030ce18d31ea424492d822fc33b185ae in term 1.
I20260925 13:10:57.737334 17836 catalog_manager.cc:5495] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 030ce18d31ea424492d822fc33b185ae (127.0.206.66). New cstate: current_term: 1 leader_uuid: "030ce18d31ea424492d822fc33b185ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a390ef3f9ebc4a89a76d72987eea1517" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da5d928a2c9c4399baa9edb532cc6214" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 40383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "030ce18d31ea424492d822fc33b185ae" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44501 } health_report { overall_health: HEALTHY } } }
I20260925 13:10:57.756116 18311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.753543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58786 (local address 127.0.206.66:44501)
0925 13:10:57.753608 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:57.753610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:57.753618 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:10:57.753656 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:57.753656 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:57.753676 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:57.753759 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:10:57.753903 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.754379 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.755295 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.755493 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.755591 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.755762 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:10:57.755867 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:10:57.755962 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:57.755980 (+ 18us) server_negotiation.cc:299] Negotiation successful
0925 13:10:57.755999 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 13:10:57.757735 17981 raft_consensus.cc:1261] T 20d28ffc0eb74671965ef80c1b36329e P a390ef3f9ebc4a89a76d72987eea1517 [term 1 FOLLOWER]: Refusing update from remote peer 030ce18d31ea424492d822fc33b185ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:10:57.757754 18249 raft_consensus.cc:1261] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214 [term 1 FOLLOWER]: Refusing update from remote peer 030ce18d31ea424492d822fc33b185ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:10:57.758091 18322 consensus_queue.cc:1035] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a390ef3f9ebc4a89a76d72987eea1517" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:10:57.758219 18318 consensus_queue.cc:1035] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "da5d928a2c9c4399baa9edb532cc6214" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 40383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:10:57.760156 18325 mvcc.cc:205] Tried to move back new op lower bound from 7333240249373286400 to 7333240249289392128. Current Snapshot: MvccSnapshot[applied={T|T < 7333240249373286400}]
I20260925 13:10:57.760617 18326 mvcc.cc:205] Tried to move back new op lower bound from 7333240249373286400 to 7333240249289392128. Current Snapshot: MvccSnapshot[applied={T|T < 7333240249373286400}]
I20260925 13:10:57.762625 17836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:10:57.762739 17836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:10:57.763469 17836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 35d31b5d2e0f41559c372340ab385365: Sending DeleteTablet for 3 replicas of tablet 20d28ffc0eb74671965ef80c1b36329e
I20260925 13:10:57.763796 18095 tablet_service.cc:1514] Processing DeleteTablet for tablet 20d28ffc0eb74671965ef80c1b36329e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:10:57 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:58780
I20260925 13:10:57.763846 18229 tablet_service.cc:1514] Processing DeleteTablet for tablet 20d28ffc0eb74671965ef80c1b36329e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:10:57 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:47640
I20260925 13:10:57.764032 18336 tablet_replica.cc:330] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214: stopping tablet replica
I20260925 13:10:57.764016 17961 tablet_service.cc:1514] Processing DeleteTablet for tablet 20d28ffc0eb74671965ef80c1b36329e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:10:57 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:44234
I20260925 13:10:57.764122 18336 raft_consensus.cc:2229] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:10:57.764218 18336 raft_consensus.cc:2258] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:10:57.764237 18337 tablet_replica.cc:330] T 20d28ffc0eb74671965ef80c1b36329e P a390ef3f9ebc4a89a76d72987eea1517: stopping tablet replica
I20260925 13:10:57.764459 18335 tablet_replica.cc:330] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae: stopping tablet replica
I20260925 13:10:57.764528 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 17898
I20260925 13:10:57.764545 18335 raft_consensus.cc:2229] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:10:57.764631 18335 raft_consensus.cc:2258] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:10:57.764658 18336 ts_tablet_manager.cc:1905] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:10:57.766203 18335 ts_tablet_manager.cc:1905] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:10:57.766237 18336 ts_tablet_manager.cc:1918] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:10:57.766297 18336 log.cc:1196] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/20d28ffc0eb74671965ef80c1b36329e
I20260925 13:10:57.766567 18336 ts_tablet_manager.cc:1939] T 20d28ffc0eb74671965ef80c1b36329e P da5d928a2c9c4399baa9edb532cc6214: Deleting consensus metadata
I20260925 13:10:57.767938 18335 ts_tablet_manager.cc:1918] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:10:57.768024 18335 log.cc:1196] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/20d28ffc0eb74671965ef80c1b36329e
I20260925 13:10:57.768257 18335 ts_tablet_manager.cc:1939] T 20d28ffc0eb74671965ef80c1b36329e P 030ce18d31ea424492d822fc33b185ae: Deleting consensus metadata
I20260925 13:10:57.768702 17823 catalog_manager.cc:4841] TS 030ce18d31ea424492d822fc33b185ae (127.0.206.66:44501): tablet 20d28ffc0eb74671965ef80c1b36329e (table test-table [id=4b3747c5fc14499b991c07f9eae5bdbb]) successfully deleted
I20260925 13:10:57.771063 17820 catalog_manager.cc:4841] TS da5d928a2c9c4399baa9edb532cc6214 (127.0.206.67:40383): tablet 20d28ffc0eb74671965ef80c1b36329e (table test-table [id=4b3747c5fc14499b991c07f9eae5bdbb]) successfully deleted
W20260925 13:10:57.771430 17822 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:32973 (error 108)
I20260925 13:10:57.771768 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 18032
I20260925 13:10:57.772428 18307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.771839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:32973 (local address 127.0.0.1:44244)
0925 13:10:57.772354 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:57.772402 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:32973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437}
W20260925 13:10:57.772574 17822 catalog_manager.cc:4584] TS a390ef3f9ebc4a89a76d72987eea1517 (127.0.206.65:32973): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 20d28ffc0eb74671965ef80c1b36329e: Network error: Client connection negotiation failed: client connection to 127.0.206.65:32973: connect: Connection refused (error 111)
I20260925 13:10:57.776187 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 18166
I20260925 13:10:57.780333 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 17804
2026-09-25T13:10:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2118 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-25T13:10:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:10:57Z Disabled control of system clock
I20260925 13:10:57.807432 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:46361
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:46361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 13:10:57.888093 18342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:10:57.888301 18342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:10:57.888321 18342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:10:57.889472 18342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:10:57.889516 18342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:10:57.889529 18342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:10:57.889541 18342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:10:57.889555 18342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:10:57.890895 18342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:46361
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:46361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:10:57.891160 18342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:10:57.891456 18342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:10:57.896121 18348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:57.896129 18350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:57.896128 18347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:57.896325 18342 server_base.cc:1034] running on GCE node
I20260925 13:10:57.896574 18342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:10:57.896869 18342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:10:57.898041 18342 hybrid_clock.cc:648] HybridClock initialized: now 1790341857898033 us; error 35 us; skew 500 ppm
I20260925 13:10:57.899317 18342 webserver.cc:460] Webserver started at http://127.0.206.126:40845/ using document root <none> and password file <none>
I20260925 13:10:57.899570 18342 fs_manager.cc:359] Metadata directory not provided
I20260925 13:10:57.899641 18342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:10:57.899787 18342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:10:57.900885 18342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "5d88dc969da949439f2a3518776d685e"
format_stamp: "Formatted at 2026-09-25 13:10:57 on dist-test-slave-chkf"
server_key: "a9413e8c750525d061a9a7b5b24462e3"
server_key_iv: "f35e8083fce9e279ed8c211f002fdbb8"
server_key_version: "encryptionkey@0"
I20260925 13:10:57.901255 18342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "5d88dc969da949439f2a3518776d685e"
format_stamp: "Formatted at 2026-09-25 13:10:57 on dist-test-slave-chkf"
server_key: "a9413e8c750525d061a9a7b5b24462e3"
server_key_iv: "f35e8083fce9e279ed8c211f002fdbb8"
server_key_version: "encryptionkey@0"
I20260925 13:10:57.902530 18342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:10:57.903309 18356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:57.903460 18342 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:57.903584 18342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "5d88dc969da949439f2a3518776d685e"
format_stamp: "Formatted at 2026-09-25 13:10:57 on dist-test-slave-chkf"
server_key: "a9413e8c750525d061a9a7b5b24462e3"
server_key_iv: "f35e8083fce9e279ed8c211f002fdbb8"
server_key_version: "encryptionkey@0"
I20260925 13:10:57.903656 18342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:10:57.925083 18342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:10:57.925420 18342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:10:57.925585 18342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:10:57.929530 18342 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:46361
I20260925 13:10:57.929577 18408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:46361 every 8 connection(s)
I20260925 13:10:57.929929 18342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:10:57.930944 18409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:10:57.931672 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 18342
I20260925 13:10:57.931787 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:10:57.931973 825 external_mini_cluster.cc:1428] Setting key 836b14a65f2f0ffa4b838d9f986e48c9
I20260925 13:10:57.933805 18409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e: Bootstrap starting.
I20260925 13:10:57.934369 18409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e: Neither blocks nor log segments found. Creating new log.
I20260925 13:10:57.934648 18409 log.cc:824] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e: Log is configured to *not* fsync() on all Append() calls
I20260925 13:10:57.935320 18409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e: No bootstrap required, opened a new log
I20260925 13:10:57.937275 18409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d88dc969da949439f2a3518776d685e" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46361 } }
I20260925 13:10:57.937433 18412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:57.932480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55318 (local address 127.0.206.126:46361)
0925 13:10:57.932704 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:57.932712 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:57.932736 (+ 24us) server_negotiation.cc:407] Connection header received
0925 13:10:57.933152 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:57.933158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:57.933216 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:57.933365 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:57.933605 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.934559 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.935380 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:57.935856 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:57.936010 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:57.936765 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:57.936770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:57.936771 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:10:57.936936 (+ 165us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:57.936975 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:10:57.936985 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:57.937069 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:57.937145 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 13:10:57.937221 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260925 13:10:57.937516 18409 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:10:57.937613 18409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d88dc969da949439f2a3518776d685e, State: Initialized, Role: FOLLOWER
I20260925 13:10:57.937742 18409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d88dc969da949439f2a3518776d685e" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46361 } }
I20260925 13:10:57.937822 18409 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:10:57.937862 18409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:10:57.937901 18409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:10:57.938467 18409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d88dc969da949439f2a3518776d685e" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46361 } }
I20260925 13:10:57.938589 18409 leader_election.cc:302] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d88dc969da949439f2a3518776d685e; no voters:
I20260925 13:10:57.938786 18409 leader_election.cc:288] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:10:57.938831 18414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:10:57.938990 18414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e [term 1 LEADER]: Becoming Leader. State: Replica: 5d88dc969da949439f2a3518776d685e, State: Running, Role: LEADER
I20260925 13:10:57.939054 18409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:10:57.939110 18414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d88dc969da949439f2a3518776d685e" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46361 } }
I20260925 13:10:57.939459 18415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d88dc969da949439f2a3518776d685e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d88dc969da949439f2a3518776d685e" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46361 } } }
I20260925 13:10:57.939570 18415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e [sys.catalog]: This master's current role is: LEADER
I20260925 13:10:57.939463 18416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d88dc969da949439f2a3518776d685e. Latest consensus state: current_term: 1 leader_uuid: "5d88dc969da949439f2a3518776d685e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d88dc969da949439f2a3518776d685e" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46361 } } }
I20260925 13:10:57.939661 18416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e [sys.catalog]: This master's current role is: LEADER
W20260925 13:10:57.940737 18430 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:10:57.940820 18430 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:10:57.940889 18423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:10:57.941072 18423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:10:57.942394 18423 catalog_manager.cc:1347] Generated new cluster ID: 4f0568b7dae346d08bc90a9d6ea4caaa
I20260925 13:10:57.942454 18423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:10:57.952759 18423 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:10:57.953228 18423 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:10:57.962544 18423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e: Generated new TSK 0
I20260925 13:10:57.962759 18423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:10:57.970062 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:46361
--builtin_ntp_servers=127.0.206.84:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 13:10:58.073318 18433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:10:58.073508 18433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:10:58.073529 18433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:10:58.074651 18433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:10:58.074687 18433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:10:58.074779 18433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:10:58.076162 18433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:46361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:10:58.076402 18433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:10:58.076706 18433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:10:58.082652 18441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:58.082665 18433 server_base.cc:1034] running on GCE node
W20260925 13:10:58.082643 18439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:58.082643 18438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:58.083062 18433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:10:58.083315 18433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:10:58.084528 18433 hybrid_clock.cc:648] HybridClock initialized: now 1790341858084510 us; error 42 us; skew 500 ppm
I20260925 13:10:58.085954 18433 webserver.cc:460] Webserver started at http://127.0.206.65:36137/ using document root <none> and password file <none>
I20260925 13:10:58.086210 18433 fs_manager.cc:359] Metadata directory not provided
I20260925 13:10:58.086265 18433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:10:58.086387 18433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:10:58.087714 18433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "97c545e61f034bc1ad647e5a19025b4a"
format_stamp: "Formatted at 2026-09-25 13:10:58 on dist-test-slave-chkf"
server_key: "b34658a01c73da9ade9b0c152bf5199f"
server_key_iv: "c178e03bfba09ac232e35d30b838f5ad"
server_key_version: "encryptionkey@0"
I20260925 13:10:58.088073 18433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "97c545e61f034bc1ad647e5a19025b4a"
format_stamp: "Formatted at 2026-09-25 13:10:58 on dist-test-slave-chkf"
server_key: "b34658a01c73da9ade9b0c152bf5199f"
server_key_iv: "c178e03bfba09ac232e35d30b838f5ad"
server_key_version: "encryptionkey@0"
I20260925 13:10:58.089556 18433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:10:58.090356 18447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:58.090541 18433 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:10:58.090639 18433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "97c545e61f034bc1ad647e5a19025b4a"
format_stamp: "Formatted at 2026-09-25 13:10:58 on dist-test-slave-chkf"
server_key: "b34658a01c73da9ade9b0c152bf5199f"
server_key_iv: "c178e03bfba09ac232e35d30b838f5ad"
server_key_version: "encryptionkey@0"
I20260925 13:10:58.090758 18433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:10:58.112241 18433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:10:58.112557 18433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:10:58.112707 18433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:10:58.112977 18433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:10:58.113335 18433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:10:58.113369 18433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:58.113391 18433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:10:58.113440 18433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:58.119050 18433 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:33527
I20260925 13:10:58.119148 18560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:33527 every 8 connection(s)
I20260925 13:10:58.119472 18433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:10:58.124876 18412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.120243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:56815 (local address 127.0.206.126:46361)
0925 13:10:58.120309 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:58.120311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:58.120553 (+ 242us) server_negotiation.cc:407] Connection header received
0925 13:10:58.121112 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:58.121113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:58.121126 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:58.121179 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:58.121871 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.122391 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.123450 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.123674 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.123763 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:58.124586 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:58.124588 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:58.124588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:10:58.124594 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:58.124608 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:10:58.124612 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:58.124667 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:58.124762 (+ 95us) server_negotiation.cc:299] Negotiation successful
0925 13:10:58.124793 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 13:10:58.125114 18563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.120232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:46361 (local address 127.0.206.65:56815)
0925 13:10:58.120429 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:58.120463 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:58.120983 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:58.121251 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:58.121254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:58.121262 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:10:58.121764 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:58.121768 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.122486 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.122487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:58.123347 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:58.123350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.123771 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.123772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:58.123911 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:58.124395 (+ 484us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:10:58.124396 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:10:58.124425 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:10:58.124447 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:10:58.124671 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:10:58.124676 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:10:58.124679 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:10:58.124766 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 13:10:58.124867 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":99,"threads_started":1}
I20260925 13:10:58.125656 18561 heartbeater.cc:347] Connected to a master server at 127.0.206.126:46361
I20260925 13:10:58.125766 18561 heartbeater.cc:464] Registering TS with master...
I20260925 13:10:58.125787 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 18433
I20260925 13:10:58.125890 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:10:58.125970 18561 heartbeater.cc:511] Master 127.0.206.126:46361 requested a full tablet report, sending...
I20260925 13:10:58.126113 825 external_mini_cluster.cc:1428] Setting key 996c728a3659f0b0f4b1263f01df33b5
I20260925 13:10:58.126581 18373 ts_manager.cc:194] Registered new tserver with Master: 97c545e61f034bc1ad647e5a19025b4a (127.0.206.65:33527)
I20260925 13:10:58.127235 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:46361
--builtin_ntp_servers=127.0.206.84:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 13:10:58.127485 18373 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.65:56815
W20260925 13:10:58.205976 18564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:10:58.206167 18564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:10:58.206188 18564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:10:58.207383 18564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:10:58.207448 18564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:10:58.207510 18564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:10:58.208786 18564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:46361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:10:58.208958 18564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:10:58.209195 18564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:10:58.214067 18572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:58.214190 18569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:58.214301 18564 server_base.cc:1034] running on GCE node
W20260925 13:10:58.214058 18570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:58.214537 18564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:10:58.214834 18564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:10:58.215996 18564 hybrid_clock.cc:648] HybridClock initialized: now 1790341858215976 us; error 30 us; skew 500 ppm
I20260925 13:10:58.217200 18564 webserver.cc:460] Webserver started at http://127.0.206.66:39163/ using document root <none> and password file <none>
I20260925 13:10:58.217437 18564 fs_manager.cc:359] Metadata directory not provided
I20260925 13:10:58.217504 18564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:10:58.217666 18564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:10:58.218772 18564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "a8207b5c4c134352860f121c2f856ea8"
format_stamp: "Formatted at 2026-09-25 13:10:58 on dist-test-slave-chkf"
server_key: "f7a245b3ab06fc83776f0badbc4356ad"
server_key_iv: "28d20782125c1ef4587d9c1150afdbb7"
server_key_version: "encryptionkey@0"
I20260925 13:10:58.219146 18564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "a8207b5c4c134352860f121c2f856ea8"
format_stamp: "Formatted at 2026-09-25 13:10:58 on dist-test-slave-chkf"
server_key: "f7a245b3ab06fc83776f0badbc4356ad"
server_key_iv: "28d20782125c1ef4587d9c1150afdbb7"
server_key_version: "encryptionkey@0"
I20260925 13:10:58.220438 18564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:10:58.221132 18578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:58.221307 18564 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:10:58.221401 18564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "a8207b5c4c134352860f121c2f856ea8"
format_stamp: "Formatted at 2026-09-25 13:10:58 on dist-test-slave-chkf"
server_key: "f7a245b3ab06fc83776f0badbc4356ad"
server_key_iv: "28d20782125c1ef4587d9c1150afdbb7"
server_key_version: "encryptionkey@0"
I20260925 13:10:58.221513 18564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:10:58.240612 18564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:10:58.240906 18564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:10:58.241047 18564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:10:58.241271 18564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:10:58.241595 18564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:10:58.241652 18564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:58.241693 18564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:10:58.241720 18564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:58.247584 18564 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:34207
I20260925 13:10:58.247604 18691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:34207 every 8 connection(s)
I20260925 13:10:58.247926 18564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:10:58.252084 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 18564
I20260925 13:10:58.252218 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:10:58.252415 825 external_mini_cluster.cc:1428] Setting key dd886f99812cd6a95d45218796697c87
I20260925 13:10:58.252851 18412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.248557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:37931 (local address 127.0.206.126:46361)
0925 13:10:58.248633 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:58.248635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:58.248874 (+ 239us) server_negotiation.cc:407] Connection header received
0925 13:10:58.249330 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:58.249331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:58.249342 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:58.249394 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:58.249997 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.250466 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.251454 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.251631 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.251681 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:58.252569 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:58.252570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:58.252571 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:10:58.252576 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:58.252590 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:10:58.252595 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:58.252668 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:58.252744 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 13:10:58.252774 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 13:10:58.253079 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.248517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:46361 (local address 127.0.206.66:37931)
0925 13:10:58.248724 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:58.248778 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:58.249251 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:58.249446 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:58.249449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:58.249457 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:10:58.249931 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:58.249935 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.250557 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.250558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:58.251381 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:58.251383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.251707 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.251708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:58.251812 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:58.252380 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:10:58.252381 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:10:58.252415 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:10:58.252437 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:10:58.252669 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:10:58.252675 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:10:58.252678 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:10:58.252747 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 13:10:58.252840 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":110,"threads_started":1}
I20260925 13:10:58.253355 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:46361
--builtin_ntp_servers=127.0.206.84:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 13:10:58.253580 18692 heartbeater.cc:347] Connected to a master server at 127.0.206.126:46361
I20260925 13:10:58.253666 18692 heartbeater.cc:464] Registering TS with master...
I20260925 13:10:58.253851 18692 heartbeater.cc:511] Master 127.0.206.126:46361 requested a full tablet report, sending...
I20260925 13:10:58.254165 18373 ts_manager.cc:194] Registered new tserver with Master: a8207b5c4c134352860f121c2f856ea8 (127.0.206.66:34207)
I20260925 13:10:58.254516 18373 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.66:37931
W20260925 13:10:58.331269 18695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:10:58.331437 18695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:10:58.331458 18695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:10:58.332552 18695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:10:58.332589 18695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:10:58.332613 18695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:10:58.333912 18695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:46361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:10:58.334154 18695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:10:58.334393 18695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:10:58.339057 18703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:58.339164 18700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:58.339061 18701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:58.339195 18695 server_base.cc:1034] running on GCE node
I20260925 13:10:58.339433 18695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:10:58.339672 18695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:10:58.340826 18695 hybrid_clock.cc:648] HybridClock initialized: now 1790341858340806 us; error 29 us; skew 500 ppm
I20260925 13:10:58.341943 18695 webserver.cc:460] Webserver started at http://127.0.206.67:46223/ using document root <none> and password file <none>
I20260925 13:10:58.342180 18695 fs_manager.cc:359] Metadata directory not provided
I20260925 13:10:58.342249 18695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:10:58.342379 18695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:10:58.343410 18695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "9a34c984214d4df7af7baaeaa984151a"
format_stamp: "Formatted at 2026-09-25 13:10:58 on dist-test-slave-chkf"
server_key: "efa8a5c87c355f40f90022ae0f40c592"
server_key_iv: "cb2c0e77ffd023b715c128cf5a74fc43"
server_key_version: "encryptionkey@0"
I20260925 13:10:58.343750 18695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "9a34c984214d4df7af7baaeaa984151a"
format_stamp: "Formatted at 2026-09-25 13:10:58 on dist-test-slave-chkf"
server_key: "efa8a5c87c355f40f90022ae0f40c592"
server_key_iv: "cb2c0e77ffd023b715c128cf5a74fc43"
server_key_version: "encryptionkey@0"
I20260925 13:10:58.345013 18695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 13:10:58.345651 18709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:58.345808 18695 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:10:58.345901 18695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "9a34c984214d4df7af7baaeaa984151a"
format_stamp: "Formatted at 2026-09-25 13:10:58 on dist-test-slave-chkf"
server_key: "efa8a5c87c355f40f90022ae0f40c592"
server_key_iv: "cb2c0e77ffd023b715c128cf5a74fc43"
server_key_version: "encryptionkey@0"
I20260925 13:10:58.345973 18695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:10:58.368454 18695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:10:58.368716 18695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:10:58.368861 18695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:10:58.369061 18695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:10:58.369355 18695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:10:58.369407 18695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:58.369447 18695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:10:58.369475 18695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:58.374660 18695 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:44565
I20260925 13:10:58.374696 18822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:44565 every 8 connection(s)
I20260925 13:10:58.375030 18695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:10:58.377926 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 18695
I20260925 13:10:58.378038 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:10:58.378243 825 external_mini_cluster.cc:1428] Setting key c5828fe2561f756ad32a0884256aefb8
I20260925 13:10:58.379947 18412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.375735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:59943 (local address 127.0.206.126:46361)
0925 13:10:58.375809 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:58.375811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:58.375965 (+ 154us) server_negotiation.cc:407] Connection header received
0925 13:10:58.376364 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:58.376365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:58.376375 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:58.376429 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:58.377014 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.377465 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.378519 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.378810 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.378908 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:58.379696 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:58.379698 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:58.379698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:10:58.379703 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:58.379717 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:10:58.379721 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:58.379776 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:58.379846 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 13:10:58.379873 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 13:10:58.380153 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.375691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:46361 (local address 127.0.206.67:59943)
0925 13:10:58.375864 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:58.375892 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:58.376292 (+ 400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:58.376489 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:58.376492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:58.376499 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:10:58.376948 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:58.376952 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.377535 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.377535 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:58.378431 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:58.378434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.379053 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.379054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:58.379158 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:58.379550 (+ 392us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:10:58.379551 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:10:58.379576 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:10:58.379606 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:10:58.379779 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:10:58.379784 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:10:58.379786 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:10:58.379849 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 13:10:58.379939 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260925 13:10:58.380599 18823 heartbeater.cc:347] Connected to a master server at 127.0.206.126:46361
I20260925 13:10:58.380666 18823 heartbeater.cc:464] Registering TS with master...
I20260925 13:10:58.380848 18823 heartbeater.cc:511] Master 127.0.206.126:46361 requested a full tablet report, sending...
I20260925 13:10:58.381178 18373 ts_manager.cc:194] Registered new tserver with Master: 9a34c984214d4df7af7baaeaa984151a (127.0.206.67:44565)
I20260925 13:10:58.381484 18373 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.67:59943
I20260925 13:10:58.389859 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:10:58.394007 18412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.391137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55322 (local address 127.0.206.126:46361)
0925 13:10:58.391217 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:58.391219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:58.391319 (+ 100us) server_negotiation.cc:407] Connection header received
0925 13:10:58.391396 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:58.391397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:58.391406 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:58.391459 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:58.391628 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.392100 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.392828 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.393029 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.393099 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:58.393786 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:58.393787 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:58.393787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:10:58.393792 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:58.393804 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:10:58.393808 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:58.393854 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:58.393909 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 13:10:58.393929 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 13:10:58.395586 18373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55322:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:10:58.397071 18373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:10:58.402951 18833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.399161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:44565 (local address 127.0.0.1:54312)
0925 13:10:58.399531 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:58.399538 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:58.399571 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:58.399748 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:58.399749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:58.399750 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:10:58.399818 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:58.399819 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.401143 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.401144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:58.401873 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:58.401875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.402644 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.402645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:58.402741 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:58.402756 (+ 15us) client_negotiation.cc:769] Sending connection context
0925 13:10:58.402832 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 13:10:58.402878 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":8,"thread_start_us":107,"threads_started":1}
I20260925 13:10:58.402957 18835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.399389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54312 (local address 127.0.206.67:44565)
0925 13:10:58.399548 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:58.399553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:58.399576 (+ 23us) server_negotiation.cc:407] Connection header received
0925 13:10:58.399633 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:58.399634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:58.399647 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:58.399721 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:10:58.399910 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.401057 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.401966 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.402564 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.402635 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:58.402738 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:58.402845 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 13:10:58.402878 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":73,"threads_started":1}
I20260925 13:10:58.403473 18832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.399137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:34207 (local address 127.0.0.1:49194)
0925 13:10:58.399323 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:58.399362 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:58.399429 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:58.400044 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:58.400046 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:58.400047 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:10:58.400102 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:58.400104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.400938 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.400939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:58.402231 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:58.402233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.403317 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.403318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:58.403362 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:58.403365 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:10:58.403401 (+ 36us) client_negotiation.cc:240] Negotiation successful
0925 13:10:58.403427 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":33,"thread_start_us":50,"threads_started":1}
I20260925 13:10:58.403650 18757 tablet_service.cc:1467] Processing CreateTablet for tablet 38c5ec70c70a4237ab89ba0d1ff7af4b (DEFAULT_TABLE table=test-table [id=12bf8f0e30dc4b63bec90855cf5ef2ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:10:58.403805 18834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.399223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49194 (local address 127.0.206.66:34207)
0925 13:10:58.399837 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:58.399841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:58.399850 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:10:58.399885 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:58.399888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:58.399902 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:58.399975 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:10:58.400161 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.400845 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.402320 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.403062 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.403133 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:58.403658 (+ 525us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:58.403685 (+ 27us) server_negotiation.cc:299] Negotiation successful
0925 13:10:58.403723 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":80,"threads_started":1}
I20260925 13:10:58.403990 18831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.399161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:33527 (local address 127.0.0.1:49932)
0925 13:10:58.399385 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:58.399394 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:58.399434 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:58.399743 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:58.399747 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:58.399749 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:10:58.399830 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:58.399831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.401114 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.401115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:58.402433 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:58.402435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.403761 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.403762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:58.403823 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:58.403826 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:10:58.403878 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 13:10:58.403915 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":7,"thread_start_us":111,"threads_started":1}
I20260925 13:10:58.404021 18757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38c5ec70c70a4237ab89ba0d1ff7af4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:10:58.404270 18836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.399393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49932 (local address 127.0.206.65:33527)
0925 13:10:58.399554 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:58.399557 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:58.399568 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:10:58.399609 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:58.399611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:58.399627 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:58.399721 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:10:58.399914 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.401027 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.402511 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.403574 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.403647 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:58.403725 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:58.404158 (+ 433us) server_negotiation.cc:299] Negotiation successful
0925 13:10:58.404199 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":71,"threads_started":1}
I20260925 13:10:58.404368 18626 tablet_service.cc:1467] Processing CreateTablet for tablet 38c5ec70c70a4237ab89ba0d1ff7af4b (DEFAULT_TABLE table=test-table [id=12bf8f0e30dc4b63bec90855cf5ef2ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:10:58.404686 18626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38c5ec70c70a4237ab89ba0d1ff7af4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:10:58.404894 18495 tablet_service.cc:1467] Processing CreateTablet for tablet 38c5ec70c70a4237ab89ba0d1ff7af4b (DEFAULT_TABLE table=test-table [id=12bf8f0e30dc4b63bec90855cf5ef2ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:10:58.405202 18495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38c5ec70c70a4237ab89ba0d1ff7af4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:10:58.407155 18837 tablet_bootstrap.cc:492] T 38c5ec70c70a4237ab89ba0d1ff7af4b P a8207b5c4c134352860f121c2f856ea8: Bootstrap starting.
I20260925 13:10:58.407874 18837 tablet_bootstrap.cc:654] T 38c5ec70c70a4237ab89ba0d1ff7af4b P a8207b5c4c134352860f121c2f856ea8: Neither blocks nor log segments found. Creating new log.
I20260925 13:10:58.407863 18838 tablet_bootstrap.cc:492] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a: Bootstrap starting.
I20260925 13:10:58.408216 18837 log.cc:824] T 38c5ec70c70a4237ab89ba0d1ff7af4b P a8207b5c4c134352860f121c2f856ea8: Log is configured to *not* fsync() on all Append() calls
I20260925 13:10:58.408411 18839 tablet_bootstrap.cc:492] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 97c545e61f034bc1ad647e5a19025b4a: Bootstrap starting.
I20260925 13:10:58.408486 18838 tablet_bootstrap.cc:654] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a: Neither blocks nor log segments found. Creating new log.
I20260925 13:10:58.408766 18838 log.cc:824] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a: Log is configured to *not* fsync() on all Append() calls
I20260925 13:10:58.409054 18839 tablet_bootstrap.cc:654] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 97c545e61f034bc1ad647e5a19025b4a: Neither blocks nor log segments found. Creating new log.
I20260925 13:10:58.409255 18837 tablet_bootstrap.cc:492] T 38c5ec70c70a4237ab89ba0d1ff7af4b P a8207b5c4c134352860f121c2f856ea8: No bootstrap required, opened a new log
I20260925 13:10:58.409328 18837 ts_tablet_manager.cc:1397] T 38c5ec70c70a4237ab89ba0d1ff7af4b P a8207b5c4c134352860f121c2f856ea8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:10:58.409364 18839 log.cc:824] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 97c545e61f034bc1ad647e5a19025b4a: Log is configured to *not* fsync() on all Append() calls
I20260925 13:10:58.409440 18838 tablet_bootstrap.cc:492] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a: No bootstrap required, opened a new log
I20260925 13:10:58.409515 18838 ts_tablet_manager.cc:1397] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:10:58.410477 18839 tablet_bootstrap.cc:492] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 97c545e61f034bc1ad647e5a19025b4a: No bootstrap required, opened a new log
I20260925 13:10:58.410544 18839 ts_tablet_manager.cc:1397] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 97c545e61f034bc1ad647e5a19025b4a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:10:58.411666 18838 raft_consensus.cc:348] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8207b5c4c134352860f121c2f856ea8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34207 } } peers { permanent_uuid: "9a34c984214d4df7af7baaeaa984151a" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44565 } } peers { permanent_uuid: "97c545e61f034bc1ad647e5a19025b4a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33527 } }
I20260925 13:10:58.411815 18838 raft_consensus.cc:374] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:10:58.411859 18838 raft_consensus.cc:729] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a34c984214d4df7af7baaeaa984151a, State: Initialized, Role: FOLLOWER
I20260925 13:10:58.411966 18838 consensus_queue.cc:260] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8207b5c4c134352860f121c2f856ea8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34207 } } peers { permanent_uuid: "9a34c984214d4df7af7baaeaa984151a" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44565 } } peers { permanent_uuid: "97c545e61f034bc1ad647e5a19025b4a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33527 } }
I20260925 13:10:58.412055 18837 raft_consensus.cc:348] T 38c5ec70c70a4237ab89ba0d1ff7af4b P a8207b5c4c134352860f121c2f856ea8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8207b5c4c134352860f121c2f856ea8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34207 } } peers { permanent_uuid: "9a34c984214d4df7af7baaeaa984151a" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44565 } } peers { permanent_uuid: "97c545e61f034bc1ad647e5a19025b4a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33527 } }
I20260925 13:10:58.412218 18837 raft_consensus.cc:374] T 38c5ec70c70a4237ab89ba0d1ff7af4b P a8207b5c4c134352860f121c2f856ea8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:10:58.412245 18838 ts_tablet_manager.cc:1428] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 13:10:58.412262 18837 raft_consensus.cc:729] T 38c5ec70c70a4237ab89ba0d1ff7af4b P a8207b5c4c134352860f121c2f856ea8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8207b5c4c134352860f121c2f856ea8, State: Initialized, Role: FOLLOWER
I20260925 13:10:58.412375 18823 heartbeater.cc:503] Master 127.0.206.126:46361 was elected leader, sending a full tablet report...
I20260925 13:10:58.412395 18837 consensus_queue.cc:260] T 38c5ec70c70a4237ab89ba0d1ff7af4b P a8207b5c4c134352860f121c2f856ea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8207b5c4c134352860f121c2f856ea8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34207 } } peers { permanent_uuid: "9a34c984214d4df7af7baaeaa984151a" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44565 } } peers { permanent_uuid: "97c545e61f034bc1ad647e5a19025b4a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33527 } }
I20260925 13:10:58.412627 18837 ts_tablet_manager.cc:1428] T 38c5ec70c70a4237ab89ba0d1ff7af4b P a8207b5c4c134352860f121c2f856ea8: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260925 13:10:58.412686 18692 heartbeater.cc:503] Master 127.0.206.126:46361 was elected leader, sending a full tablet report...
I20260925 13:10:58.413499 18839 raft_consensus.cc:348] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 97c545e61f034bc1ad647e5a19025b4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8207b5c4c134352860f121c2f856ea8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34207 } } peers { permanent_uuid: "9a34c984214d4df7af7baaeaa984151a" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44565 } } peers { permanent_uuid: "97c545e61f034bc1ad647e5a19025b4a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33527 } }
I20260925 13:10:58.413615 18839 raft_consensus.cc:374] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 97c545e61f034bc1ad647e5a19025b4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:10:58.413678 18839 raft_consensus.cc:729] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 97c545e61f034bc1ad647e5a19025b4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c545e61f034bc1ad647e5a19025b4a, State: Initialized, Role: FOLLOWER
I20260925 13:10:58.413789 18839 consensus_queue.cc:260] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 97c545e61f034bc1ad647e5a19025b4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8207b5c4c134352860f121c2f856ea8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34207 } } peers { permanent_uuid: "9a34c984214d4df7af7baaeaa984151a" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44565 } } peers { permanent_uuid: "97c545e61f034bc1ad647e5a19025b4a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33527 } }
I20260925 13:10:58.414021 18839 ts_tablet_manager.cc:1428] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 97c545e61f034bc1ad647e5a19025b4a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 13:10:58.414112 18561 heartbeater.cc:503] Master 127.0.206.126:46361 was elected leader, sending a full tablet report...
I20260925 13:10:58.432751 18843 raft_consensus.cc:482] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:10:58.432837 18843 raft_consensus.cc:504] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8207b5c4c134352860f121c2f856ea8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34207 } } peers { permanent_uuid: "9a34c984214d4df7af7baaeaa984151a" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44565 } } peers { permanent_uuid: "97c545e61f034bc1ad647e5a19025b4a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33527 } }
I20260925 13:10:58.433193 18843 leader_election.cc:288] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8207b5c4c134352860f121c2f856ea8 (127.0.206.66:34207), 97c545e61f034bc1ad647e5a19025b4a (127.0.206.65:33527)
I20260925 13:10:58.435892 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.433318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:34207 (local address 127.0.206.67:45685)
0925 13:10:58.433375 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:58.433387 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:58.433433 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:58.433601 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:58.433602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:58.433604 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:10:58.433681 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:58.433683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.434329 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.434330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:58.435080 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:58.435083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.435644 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.435645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:58.435718 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:58.435721 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:10:58.435783 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 13:10:58.435823 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 13:10:58.435894 18834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.433400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:45685 (local address 127.0.206.66:34207)
0925 13:10:58.433441 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:58.433443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:58.433475 (+ 32us) server_negotiation.cc:407] Connection header received
0925 13:10:58.433501 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:58.433502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:58.433510 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:58.433584 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:10:58.433757 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.434234 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.435198 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.435542 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.435645 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:58.435653 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:58.435802 (+ 149us) server_negotiation.cc:299] Negotiation successful
0925 13:10:58.435836 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 13:10:58.436190 18646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c5ec70c70a4237ab89ba0d1ff7af4b" candidate_uuid: "9a34c984214d4df7af7baaeaa984151a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8207b5c4c134352860f121c2f856ea8" is_pre_election: true
I20260925 13:10:58.436321 18836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.433431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:40523 (local address 127.0.206.65:33527)
0925 13:10:58.433468 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:58.433470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:58.433661 (+ 191us) server_negotiation.cc:407] Connection header received
0925 13:10:58.433723 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:58.433724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:58.433736 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:58.433807 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:10:58.433955 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.434399 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.435438 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.435792 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.435871 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:58.435879 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:58.436184 (+ 305us) server_negotiation.cc:299] Negotiation successful
0925 13:10:58.436250 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 13:10:58.436337 18646 raft_consensus.cc:2454] T 38c5ec70c70a4237ab89ba0d1ff7af4b P a8207b5c4c134352860f121c2f856ea8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a34c984214d4df7af7baaeaa984151a in term 0.
I20260925 13:10:58.436544 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.433386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:33527 (local address 127.0.206.67:40523)
0925 13:10:58.433585 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:58.433597 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:58.433665 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:58.433809 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:58.433810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:58.433811 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:10:58.433883 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:58.433884 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.434504 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.434505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:58.435353 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:58.435355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.435891 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.435893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:58.435987 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:58.435992 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 13:10:58.436427 (+ 435us) client_negotiation.cc:240] Negotiation successful
0925 13:10:58.436473 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":151,"threads_started":1}
I20260925 13:10:58.436542 18713 leader_election.cc:302] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a34c984214d4df7af7baaeaa984151a, a8207b5c4c134352860f121c2f856ea8; no voters:
I20260925 13:10:58.436700 18843 raft_consensus.cc:2781] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:10:58.436746 18843 raft_consensus.cc:482] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:10:58.436772 18843 raft_consensus.cc:3037] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:10:58.436893 18515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c5ec70c70a4237ab89ba0d1ff7af4b" candidate_uuid: "9a34c984214d4df7af7baaeaa984151a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c545e61f034bc1ad647e5a19025b4a" is_pre_election: true
I20260925 13:10:58.437031 18515 raft_consensus.cc:2454] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 97c545e61f034bc1ad647e5a19025b4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a34c984214d4df7af7baaeaa984151a in term 0.
I20260925 13:10:58.437561 18843 raft_consensus.cc:504] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8207b5c4c134352860f121c2f856ea8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34207 } } peers { permanent_uuid: "9a34c984214d4df7af7baaeaa984151a" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44565 } } peers { permanent_uuid: "97c545e61f034bc1ad647e5a19025b4a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33527 } }
I20260925 13:10:58.437691 18843 leader_election.cc:288] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [CANDIDATE]: Term 1 election: Requested vote from peers a8207b5c4c134352860f121c2f856ea8 (127.0.206.66:34207), 97c545e61f034bc1ad647e5a19025b4a (127.0.206.65:33527)
I20260925 13:10:58.437824 18646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c5ec70c70a4237ab89ba0d1ff7af4b" candidate_uuid: "9a34c984214d4df7af7baaeaa984151a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8207b5c4c134352860f121c2f856ea8"
I20260925 13:10:58.437888 18515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c5ec70c70a4237ab89ba0d1ff7af4b" candidate_uuid: "9a34c984214d4df7af7baaeaa984151a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97c545e61f034bc1ad647e5a19025b4a"
I20260925 13:10:58.437901 18646 raft_consensus.cc:3037] T 38c5ec70c70a4237ab89ba0d1ff7af4b P a8207b5c4c134352860f121c2f856ea8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:10:58.437964 18515 raft_consensus.cc:3037] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 97c545e61f034bc1ad647e5a19025b4a [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:10:58.438648 18646 raft_consensus.cc:2454] T 38c5ec70c70a4237ab89ba0d1ff7af4b P a8207b5c4c134352860f121c2f856ea8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a34c984214d4df7af7baaeaa984151a in term 1.
I20260925 13:10:58.438735 18515 raft_consensus.cc:2454] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 97c545e61f034bc1ad647e5a19025b4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a34c984214d4df7af7baaeaa984151a in term 1.
I20260925 13:10:58.438840 18713 leader_election.cc:302] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a34c984214d4df7af7baaeaa984151a, a8207b5c4c134352860f121c2f856ea8; no voters:
I20260925 13:10:58.438952 18843 raft_consensus.cc:2781] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:10:58.439082 18843 raft_consensus.cc:686] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [term 1 LEADER]: Becoming Leader. State: Replica: 9a34c984214d4df7af7baaeaa984151a, State: Running, Role: LEADER
I20260925 13:10:58.439191 18843 consensus_queue.cc:237] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8207b5c4c134352860f121c2f856ea8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34207 } } peers { permanent_uuid: "9a34c984214d4df7af7baaeaa984151a" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44565 } } peers { permanent_uuid: "97c545e61f034bc1ad647e5a19025b4a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33527 } }
I20260925 13:10:58.439992 18373 catalog_manager.cc:5495] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a34c984214d4df7af7baaeaa984151a (127.0.206.67). New cstate: current_term: 1 leader_uuid: "9a34c984214d4df7af7baaeaa984151a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8207b5c4c134352860f121c2f856ea8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a34c984214d4df7af7baaeaa984151a" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 44565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97c545e61f034bc1ad647e5a19025b4a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260925 13:10:58.452455 18835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.450382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54318 (local address 127.0.206.67:44565)
0925 13:10:58.450434 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:58.450436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:58.450450 (+ 14us) server_negotiation.cc:407] Connection header received
0925 13:10:58.450475 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:58.450476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:58.450489 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:58.450545 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:10:58.450651 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.451116 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.451792 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:58.451957 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:58.452004 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:58.452173 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:10:58.452254 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:10:58.452310 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:58.452346 (+ 36us) server_negotiation.cc:299] Negotiation successful
0925 13:10:58.452359 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 13:10:58.454066 18515 raft_consensus.cc:1261] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 97c545e61f034bc1ad647e5a19025b4a [term 1 FOLLOWER]: Refusing update from remote peer 9a34c984214d4df7af7baaeaa984151a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:10:58.454078 18646 raft_consensus.cc:1261] T 38c5ec70c70a4237ab89ba0d1ff7af4b P a8207b5c4c134352860f121c2f856ea8 [term 1 FOLLOWER]: Refusing update from remote peer 9a34c984214d4df7af7baaeaa984151a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:10:58.454425 18847 consensus_queue.cc:1035] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c545e61f034bc1ad647e5a19025b4a" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:10:58.454548 18843 consensus_queue.cc:1035] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8207b5c4c134352860f121c2f856ea8" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:10:58.456616 18850 mvcc.cc:205] Tried to move back new op lower bound from 7333240252225384448 to 7333240252167643136. Current Snapshot: MvccSnapshot[applied={T|T < 7333240252225384448}]
I20260925 13:10:58.457053 18851 mvcc.cc:205] Tried to move back new op lower bound from 7333240252225384448 to 7333240252167643136. Current Snapshot: MvccSnapshot[applied={T|T < 7333240252225384448}]
I20260925 13:10:58.457191 18852 mvcc.cc:205] Tried to move back new op lower bound from 7333240252225384448 to 7333240252167643136. Current Snapshot: MvccSnapshot[applied={T|T < 7333240252225384448}]
I20260925 13:10:58.458829 18372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:10:58.458916 18372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:10:58.459625 18372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d88dc969da949439f2a3518776d685e: Sending DeleteTablet for 3 replicas of tablet 38c5ec70c70a4237ab89ba0d1ff7af4b
I20260925 13:10:58.459913 18757 tablet_service.cc:1514] Processing DeleteTablet for tablet 38c5ec70c70a4237ab89ba0d1ff7af4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:10:58 UTC) from {username='slave'} at 127.0.0.1:54312
I20260925 13:10:58.459935 18626 tablet_service.cc:1514] Processing DeleteTablet for tablet 38c5ec70c70a4237ab89ba0d1ff7af4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:10:58 UTC) from {username='slave'} at 127.0.0.1:49194
I20260925 13:10:58.459939 18495 tablet_service.cc:1514] Processing DeleteTablet for tablet 38c5ec70c70a4237ab89ba0d1ff7af4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:10:58 UTC) from {username='slave'} at 127.0.0.1:49932
I20260925 13:10:58.460184 18860 tablet_replica.cc:330] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a: stopping tablet replica
I20260925 13:10:58.460290 18860 raft_consensus.cc:2229] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:10:58.460311 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 18433
I20260925 13:10:58.460398 18860 raft_consensus.cc:2258] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:10:58.461570 18860 ts_tablet_manager.cc:1905] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:10:58.462975 18860 ts_tablet_manager.cc:1918] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:10:58.463096 18860 log.cc:1196] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/38c5ec70c70a4237ab89ba0d1ff7af4b
I20260925 13:10:58.463376 18860 ts_tablet_manager.cc:1939] T 38c5ec70c70a4237ab89ba0d1ff7af4b P 9a34c984214d4df7af7baaeaa984151a: Deleting consensus metadata
I20260925 13:10:58.463780 18358 catalog_manager.cc:4841] TS 9a34c984214d4df7af7baaeaa984151a (127.0.206.67:44565): tablet 38c5ec70c70a4237ab89ba0d1ff7af4b (table test-table [id=12bf8f0e30dc4b63bec90855cf5ef2ed]) successfully deleted
W20260925 13:10:58.465462 18357 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:33527 (error 108)
I20260925 13:10:58.465646 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 18564
I20260925 13:10:58.465924 18831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.465810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:33527 (local address 127.0.0.1:49942)
0925 13:10:58.465853 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:58.465895 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 13:10:58.466025 18357 catalog_manager.cc:4584] TS 97c545e61f034bc1ad647e5a19025b4a (127.0.206.65:33527): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 38c5ec70c70a4237ab89ba0d1ff7af4b: Network error: Client connection negotiation failed: client connection to 127.0.206.65:33527: connect: Connection refused (error 111)
I20260925 13:10:58.473676 18831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:58.473557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:34207 (local address 127.0.0.1:49204)
0925 13:10:58.473614 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:58.473642 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.66:34207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 13:10:58.473799 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 18695
I20260925 13:10:58.478111 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 18342
2026-09-25T13:10:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (697 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:10:58 dist-test-slave-chkf krb5kdc[18866](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:10:58 dist-test-slave-chkf krb5kdc[18866](info): set up 2 sockets
Sep 25 13:10:58 dist-test-slave-chkf krb5kdc[18866](info): commencing operation
krb5kdc: starting...
W20260925 13:10:59.518445 825 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:10:59 dist-test-slave-chkf krb5kdc[18866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:10:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:10:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:10:59.627610 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:42597
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:42597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:10:59.704913 18879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:10:59.705090 18879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:10:59.705111 18879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:10:59.706140 18879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:10:59.706174 18879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:10:59.706188 18879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:10:59.706200 18879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:10:59.706212 18879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:10:59.707629 18879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:42597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:42597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:10:59.707791 18879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:10:59.707945 18879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:10:59.712736 18885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:59.712767 18879 server_base.cc:1034] running on GCE node
W20260925 13:10:59.712781 18887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:59.712770 18884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:59.713243 18879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:10:59.713474 18879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:10:59.714617 18879 hybrid_clock.cc:648] HybridClock initialized: now 1790341859714608 us; error 32 us; skew 500 ppm
Sep 25 13:10:59 dist-test-slave-chkf krb5kdc[18866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341859, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:10:59.716971 18879 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:10:59.717381 18879 webserver.cc:460] Webserver started at http://127.0.206.126:40077/ using document root <none> and password file <none>
I20260925 13:10:59.717583 18879 fs_manager.cc:359] Metadata directory not provided
I20260925 13:10:59.717624 18879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:10:59.717707 18879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:10:59.718616 18879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "66d88665d00b4e87969f23cec74eb183"
format_stamp: "Formatted at 2026-09-25 13:10:59 on dist-test-slave-chkf"
server_key: "2dbba80afe8d9ea015f0ac8b46f264ef"
server_key_iv: "d22d55cf65f9e2dafbf52a8b3f9e2940"
server_key_version: "encryptionkey@0"
I20260925 13:10:59.718966 18879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "66d88665d00b4e87969f23cec74eb183"
format_stamp: "Formatted at 2026-09-25 13:10:59 on dist-test-slave-chkf"
server_key: "2dbba80afe8d9ea015f0ac8b46f264ef"
server_key_iv: "d22d55cf65f9e2dafbf52a8b3f9e2940"
server_key_version: "encryptionkey@0"
I20260925 13:10:59.720147 18879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:10:59.720799 18894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:59.720961 18879 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:10:59.721062 18879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "66d88665d00b4e87969f23cec74eb183"
format_stamp: "Formatted at 2026-09-25 13:10:59 on dist-test-slave-chkf"
server_key: "2dbba80afe8d9ea015f0ac8b46f264ef"
server_key_iv: "d22d55cf65f9e2dafbf52a8b3f9e2940"
server_key_version: "encryptionkey@0"
I20260925 13:10:59.721131 18879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:10:59.736716 18879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:10:59.738907 18879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:10:59.739058 18879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:10:59.742975 18879 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:42597
I20260925 13:10:59.743007 18946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:42597 every 8 connection(s)
I20260925 13:10:59.743310 18879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:10:59.744082 18947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:10:59.746107 18947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183: Bootstrap starting.
I20260925 13:10:59.746624 18947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183: Neither blocks nor log segments found. Creating new log.
I20260925 13:10:59.746923 18947 log.cc:824] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183: Log is configured to *not* fsync() on all Append() calls
I20260925 13:10:59.747588 18947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183: No bootstrap required, opened a new log
I20260925 13:10:59.749209 18947 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d88665d00b4e87969f23cec74eb183" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42597 } }
I20260925 13:10:59.749332 18947 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:10:59.749353 18947 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66d88665d00b4e87969f23cec74eb183, State: Initialized, Role: FOLLOWER
I20260925 13:10:59.749534 18947 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d88665d00b4e87969f23cec74eb183" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42597 } }
I20260925 13:10:59.749612 18947 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:10:59.749653 18947 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:10:59.749701 18947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:10:59.750177 18947 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d88665d00b4e87969f23cec74eb183" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42597 } }
I20260925 13:10:59.750281 18947 leader_election.cc:302] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66d88665d00b4e87969f23cec74eb183; no voters:
I20260925 13:10:59.750449 18947 leader_election.cc:288] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:10:59.750487 18950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:10:59.750663 18950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183 [term 1 LEADER]: Becoming Leader. State: Replica: 66d88665d00b4e87969f23cec74eb183, State: Running, Role: LEADER
I20260925 13:10:59.750684 18947 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:10:59.750775 18950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d88665d00b4e87969f23cec74eb183" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42597 } }
I20260925 13:10:59.751317 18952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66d88665d00b4e87969f23cec74eb183. Latest consensus state: current_term: 1 leader_uuid: "66d88665d00b4e87969f23cec74eb183" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d88665d00b4e87969f23cec74eb183" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42597 } } }
I20260925 13:10:59.751405 18952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183 [sys.catalog]: This master's current role is: LEADER
I20260925 13:10:59.751647 18951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66d88665d00b4e87969f23cec74eb183" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d88665d00b4e87969f23cec74eb183" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42597 } } }
I20260925 13:10:59.751721 18959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:10:59.751791 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 18879
I20260925 13:10:59.751804 18951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183 [sys.catalog]: This master's current role is: LEADER
I20260925 13:10:59.751886 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:10:59.752125 825 external_mini_cluster.cc:1428] Setting key 07918220d4a7b48a3fda86a16cd84ec5
I20260925 13:10:59.752254 18959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:10:59.753878 18959 catalog_manager.cc:1347] Generated new cluster ID: 2961c0d1206b4d3d9b76deb4717d3751
I20260925 13:10:59.753927 18959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 13:10:59 dist-test-slave-chkf krb5kdc[18866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:10:59.759969 18959 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:10:59.760561 18959 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:10:59.763955 18969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:59.752728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46010 (local address 127.0.206.126:42597)
0925 13:10:59.753357 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:59.753366 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:59.753394 (+ 28us) server_negotiation.cc:407] Connection header received
0925 13:10:59.753526 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:59.753534 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:59.753588 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:59.753725 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:59.754467 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:59.755880 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:59.756969 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:59.757593 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:59.757958 (+ 365us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:59.760414 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:59.760417 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:59.760418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:10:59.760428 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:59.762334 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:59.762740 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:59.762743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:59.762744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:59.762844 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:59.763050 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:59.763051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:59.763051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:59.763302 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:59.763453 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:59.763521 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 13:10:59.763757 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":97,"threads_started":1}
I20260925 13:10:59.765808 18959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183: Generated new TSK 0
I20260925 13:10:59.765959 18959 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260925 13:10:59.792899 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:42597
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:10:59.873261 18973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:10:59.873456 18973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:10:59.873477 18973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:10:59.874603 18973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:10:59.874647 18973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:10:59.874673 18973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:10:59.876080 18973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:42597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:10:59.876317 18973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:10:59.876550 18973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:10:59.881405 18979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:59.881373 18981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:10:59.881405 18978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:10:59.881466 18973 server_base.cc:1034] running on GCE node
I20260925 13:10:59.881724 18973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:10:59.881966 18973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:10:59.883101 18973 hybrid_clock.cc:648] HybridClock initialized: now 1790341859883089 us; error 27 us; skew 500 ppm
Sep 25 13:10:59 dist-test-slave-chkf krb5kdc[18866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341859, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:10:59.885532 18973 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:10:59.886029 18973 webserver.cc:460] Webserver started at http://127.0.206.65:36251/ using document root <none> and password file <none>
I20260925 13:10:59.886201 18973 fs_manager.cc:359] Metadata directory not provided
I20260925 13:10:59.886240 18973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:10:59.886322 18973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:10:59.887295 18973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "995080c25374464fb17d7ba0b4a11eb8"
format_stamp: "Formatted at 2026-09-25 13:10:59 on dist-test-slave-chkf"
server_key: "1b59a867c601465e6997a6a28a813e49"
server_key_iv: "870da9fca936dba07680a15276ded2a1"
server_key_version: "encryptionkey@0"
I20260925 13:10:59.887615 18973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "995080c25374464fb17d7ba0b4a11eb8"
format_stamp: "Formatted at 2026-09-25 13:10:59 on dist-test-slave-chkf"
server_key: "1b59a867c601465e6997a6a28a813e49"
server_key_iv: "870da9fca936dba07680a15276ded2a1"
server_key_version: "encryptionkey@0"
I20260925 13:10:59.888780 18973 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:10:59.889385 18988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:59.889535 18973 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:10:59.889642 18973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "995080c25374464fb17d7ba0b4a11eb8"
format_stamp: "Formatted at 2026-09-25 13:10:59 on dist-test-slave-chkf"
server_key: "1b59a867c601465e6997a6a28a813e49"
server_key_iv: "870da9fca936dba07680a15276ded2a1"
server_key_version: "encryptionkey@0"
I20260925 13:10:59.889714 18973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:10:59.909463 18973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:10:59.911653 18973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:10:59.911806 18973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:10:59.912024 18973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:10:59.912302 18973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:10:59.912354 18973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:59.912385 18973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:10:59.912412 18973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:10:59.917960 18973 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:43223
I20260925 13:10:59.918010 19101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:43223 every 8 connection(s)
I20260925 13:10:59.918278 18973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
Sep 25 13:10:59 dist-test-slave-chkf krb5kdc[18866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341859, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:10:59.926910 18969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:59.918946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:59539 (local address 127.0.206.126:42597)
0925 13:10:59.918985 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 13:10:59.918987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:10:59.919225 (+ 238us) server_negotiation.cc:407] Connection header received
0925 13:10:59.920112 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:10:59.920113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:10:59.920124 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:10:59.920180 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:10:59.920755 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:59.921251 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:59.922220 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:10:59.922420 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:10:59.922491 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:59.924443 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:10:59.924444 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:10:59.924444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:10:59.924450 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:10:59.925815 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:59.926143 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:59.926143 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:59.926144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:59.926192 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:10:59.926387 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:10:59.926387 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:10:59.926387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:10:59.926479 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:10:59.926526 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 13:10:59.926782 (+ 256us) server_negotiation.cc:299] Negotiation successful
0925 13:10:59.926847 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:10:59.927090 19104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:10:59.918937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:42597 (local address 127.0.206.65:59539)
0925 13:10:59.919117 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 13:10:59.919147 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 13:10:59.919999 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:10:59.920244 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:10:59.920248 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:10:59.920483 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:10:59.920654 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:59.920658 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:59.921341 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:59.921342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:59.922121 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:10:59.922123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:10:59.922509 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:10:59.922510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:10:59.922626 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:10:59.923081 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:10:59.923088 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:10:59.924364 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:10:59.925880 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:59.925885 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:59.925887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:59.926079 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:59.926246 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:10:59.926246 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:10:59.926247 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:10:59.926334 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:10:59.926529 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:10:59.926533 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:10:59.926636 (+ 103us) client_negotiation.cc:769] Sending connection context
0925 13:10:59.926754 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 13:10:59.926874 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260925 13:10:59.927145 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 18973
I20260925 13:10:59.927256 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:10:59.927522 825 external_mini_cluster.cc:1428] Setting key 3173824dec2b6c7443bd8c88a0ab1463
I20260925 13:10:59.927670 19102 heartbeater.cc:347] Connected to a master server at 127.0.206.126:42597
I20260925 13:10:59.927778 19102 heartbeater.cc:464] Registering TS with master...
I20260925 13:10:59.927965 19102 heartbeater.cc:511] Master 127.0.206.126:42597 requested a full tablet report, sending...
I20260925 13:10:59.928445 18911 ts_manager.cc:194] Registered new tserver with Master: 995080c25374464fb17d7ba0b4a11eb8 (127.0.206.65:43223)
I20260925 13:10:59.929031 18911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:59539
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260925 13:10:59.951267 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:42597
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:00.030009 19107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:00.030196 19107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:00.030220 19107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:00.031383 19107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:00.031424 19107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:00.031450 19107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:11:00.032923 19107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:42597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:00.033141 19107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:00.033352 19107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:00.038132 19113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:00.038161 19112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:00.038156 19115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:00.038301 19107 server_base.cc:1034] running on GCE node
I20260925 13:11:00.038515 19107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:00.038801 19107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:00.039940 19107 hybrid_clock.cc:648] HybridClock initialized: now 1790341860039929 us; error 30 us; skew 500 ppm
Sep 25 13:11:00 dist-test-slave-chkf krb5kdc[18866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341860, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:00.042357 19107 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:11:00.042804 19107 webserver.cc:460] Webserver started at http://127.0.206.66:35737/ using document root <none> and password file <none>
I20260925 13:11:00.043009 19107 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:00.043051 19107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:00.043177 19107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:00.044140 19107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "1de47e530bcc4ecea04dfe871cdf04be"
format_stamp: "Formatted at 2026-09-25 13:11:00 on dist-test-slave-chkf"
server_key: "b40885999c53129207b9d2129255abd5"
server_key_iv: "99a01a50de6f876e9155e7da13d905af"
server_key_version: "encryptionkey@0"
I20260925 13:11:00.044459 19107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "1de47e530bcc4ecea04dfe871cdf04be"
format_stamp: "Formatted at 2026-09-25 13:11:00 on dist-test-slave-chkf"
server_key: "b40885999c53129207b9d2129255abd5"
server_key_iv: "99a01a50de6f876e9155e7da13d905af"
server_key_version: "encryptionkey@0"
I20260925 13:11:00.045656 19107 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:11:00.046311 19122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:00.046492 19107 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:11:00.046595 19107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "1de47e530bcc4ecea04dfe871cdf04be"
format_stamp: "Formatted at 2026-09-25 13:11:00 on dist-test-slave-chkf"
server_key: "b40885999c53129207b9d2129255abd5"
server_key_iv: "99a01a50de6f876e9155e7da13d905af"
server_key_version: "encryptionkey@0"
I20260925 13:11:00.046663 19107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:00.071288 19107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:00.073267 19107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:00.073407 19107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:00.073653 19107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:00.073961 19107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:00.073993 19107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:00.074040 19107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:00.074069 19107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:00.079950 19107 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:44717
I20260925 13:11:00.079999 19235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:44717 every 8 connection(s)
I20260925 13:11:00.080317 19107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:11:00.085562 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 19107
I20260925 13:11:00.085714 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:11:00.085917 825 external_mini_cluster.cc:1428] Setting key 9e22afb3b67938b82d93f838b87f81ff
Sep 25 13:11:00 dist-test-slave-chkf krb5kdc[18866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341860, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:11:00.089525 18969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.081040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:39903 (local address 127.0.206.126:42597)
0925 13:11:00.081103 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:00.081105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:00.081313 (+ 208us) server_negotiation.cc:407] Connection header received
0925 13:11:00.082169 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:00.082170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:00.082181 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:00.082235 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:00.082902 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.083391 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.084345 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.084521 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.084576 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.086806 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:00.086807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:00.086807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:00.086813 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:00.088240 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:00.088610 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:00.088612 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:00.088612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:00.088676 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:00.088895 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:00.088896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:00.088896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:00.089038 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:00.089109 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:00.089352 (+ 243us) server_negotiation.cc:299] Negotiation successful
0925 13:11:00.089439 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 13:11:00.089722 19238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.080976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:42597 (local address 127.0.206.66:39903)
0925 13:11:00.081183 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:00.081217 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:00.082079 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:00.082288 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:00.082291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:00.082544 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:00.082787 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:00.082794 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.083466 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.083466 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:00.084273 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:00.084274 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.084590 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.084591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:00.084685 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.085121 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:00.085129 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:00.086698 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:00.088320 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:00.088326 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:00.088327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:00.088541 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:00.088741 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:00.088741 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:00.088742 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:00.088835 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:00.089102 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:00.089105 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:00.089218 (+ 113us) client_negotiation.cc:769] Sending connection context
0925 13:11:00.089328 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 13:11:00.089474 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":68,"threads_started":1}
I20260925 13:11:00.090165 19236 heartbeater.cc:347] Connected to a master server at 127.0.206.126:42597
I20260925 13:11:00.090251 19236 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:00.090435 19236 heartbeater.cc:511] Master 127.0.206.126:42597 requested a full tablet report, sending...
I20260925 13:11:00.090831 18911 ts_manager.cc:194] Registered new tserver with Master: 1de47e530bcc4ecea04dfe871cdf04be (127.0.206.66:44717)
I20260925 13:11:00.091362 18911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:39903
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260925 13:11:00.109676 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:42597
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:00.188004 19241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:00.188189 19241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:00.188211 19241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:00.189330 19241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:00.189370 19241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:00.189421 19241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:11:00.190863 19241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:42597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:00.191089 19241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:00.191301 19241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:00.196259 19246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:00.196293 19249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:00.196287 19247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:00.196578 19241 server_base.cc:1034] running on GCE node
I20260925 13:11:00.196780 19241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:00.197021 19241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:00.198207 19241 hybrid_clock.cc:648] HybridClock initialized: now 1790341860198186 us; error 31 us; skew 500 ppm
Sep 25 13:11:00 dist-test-slave-chkf krb5kdc[18866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341860, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:00.200579 19241 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:11:00.201011 19241 webserver.cc:460] Webserver started at http://127.0.206.67:41965/ using document root <none> and password file <none>
I20260925 13:11:00.201184 19241 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:00.201222 19241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:00.201306 19241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:00.202244 19241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "83a900634fd74f958443b436097b9ffa"
format_stamp: "Formatted at 2026-09-25 13:11:00 on dist-test-slave-chkf"
server_key: "88b04c50296449cdfda2b6cde2f96a83"
server_key_iv: "be45a84eaa95b07f9eb6ddb86e0de80a"
server_key_version: "encryptionkey@0"
I20260925 13:11:00.202533 19241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "83a900634fd74f958443b436097b9ffa"
format_stamp: "Formatted at 2026-09-25 13:11:00 on dist-test-slave-chkf"
server_key: "88b04c50296449cdfda2b6cde2f96a83"
server_key_iv: "be45a84eaa95b07f9eb6ddb86e0de80a"
server_key_version: "encryptionkey@0"
I20260925 13:11:00.203815 19241 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:11:00.204504 19256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:00.204677 19241 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:11:00.204752 19241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "83a900634fd74f958443b436097b9ffa"
format_stamp: "Formatted at 2026-09-25 13:11:00 on dist-test-slave-chkf"
server_key: "88b04c50296449cdfda2b6cde2f96a83"
server_key_iv: "be45a84eaa95b07f9eb6ddb86e0de80a"
server_key_version: "encryptionkey@0"
I20260925 13:11:00.204802 19241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:00.217186 19241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:00.219378 19241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:00.219547 19241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:00.219754 19241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:00.220041 19241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:00.220093 19241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:00.220131 19241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:00.220158 19241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:00.225812 19241 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:41597
I20260925 13:11:00.225915 19369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:41597 every 8 connection(s)
I20260925 13:11:00.226171 19241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
Sep 25 13:11:00 dist-test-slave-chkf krb5kdc[18866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341860, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:11:00.233934 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 19241
I20260925 13:11:00.234050 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:11:00.234261 825 external_mini_cluster.cc:1428] Setting key a29a667a034e63e7d7889ce7c8d340a9
I20260925 13:11:00.234992 18969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.226898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:55529 (local address 127.0.206.126:42597)
0925 13:11:00.226941 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:00.226943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:00.227144 (+ 201us) server_negotiation.cc:407] Connection header received
0925 13:11:00.227998 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:00.227998 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:00.228009 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:00.228058 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:00.228586 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.229060 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.229958 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.230133 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.230180 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.232129 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:00.232130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:00.232130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:00.232137 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:00.233497 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:00.233928 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:00.233929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:00.233930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:00.233989 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:00.234279 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:00.234280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:00.234280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:00.234412 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:00.234484 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:00.234853 (+ 369us) server_negotiation.cc:299] Negotiation successful
0925 13:11:00.234931 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 13:11:00.235241 19372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.226903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:42597 (local address 127.0.206.67:55529)
0925 13:11:00.227047 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:00.227083 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:00.227912 (+ 829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:00.228114 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:00.228117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:00.228340 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:00.228508 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:00.228511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.229133 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.229134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:00.229889 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:00.229890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.230200 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.230200 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:00.230292 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.230661 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:00.230669 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:00.232018 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:00.233609 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:00.233616 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:00.233617 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:00.233842 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:00.234068 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:00.234068 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:00.234069 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:00.234206 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:00.234488 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:00.234494 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:00.234673 (+ 179us) client_negotiation.cc:769] Sending connection context
0925 13:11:00.234812 (+ 139us) client_negotiation.cc:240] Negotiation successful
0925 13:11:00.234963 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260925 13:11:00.235726 19370 heartbeater.cc:347] Connected to a master server at 127.0.206.126:42597
I20260925 13:11:00.235821 19370 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:00.235993 19370 heartbeater.cc:511] Master 127.0.206.126:42597 requested a full tablet report, sending...
I20260925 13:11:00.236328 18911 ts_manager.cc:194] Registered new tserver with Master: 83a900634fd74f958443b436097b9ffa (127.0.206.67:41597)
I20260925 13:11:00.236670 18911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:55529
I20260925 13:11:00.245954 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:11:00.253643 18969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.247240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46014 (local address 127.0.206.126:42597)
0925 13:11:00.247320 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:00.247321 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:00.247400 (+ 79us) server_negotiation.cc:407] Connection header received
0925 13:11:00.247484 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:00.247485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:00.247495 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:00.247563 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:00.248116 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.248566 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.249415 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.249637 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.249693 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.251113 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:00.251113 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:00.251114 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:00.251119 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:00.252580 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:00.252929 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:00.252930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:00.252930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:00.253003 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:00.253182 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:00.253182 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:00.253183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:00.253296 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:00.253369 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:00.253468 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 13:11:00.253512 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 13:11:00.255091 18911 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:11:00.256548 18911 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:11:00.262585 19380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.258531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:44717 (local address 127.0.0.1:33776)
0925 13:11:00.258680 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:00.258729 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:00.258793 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:00.259070 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:00.259076 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:00.259078 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:00.259143 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:00.259145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.260446 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.260448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:00.261504 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:00.261505 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.262313 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.262314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:00.262367 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.262376 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 13:11:00.262458 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 13:11:00.262511 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":3,"thread_start_us":57,"threads_started":1}
I20260925 13:11:00.262607 19381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.258652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33776 (local address 127.0.206.66:44717)
0925 13:11:00.258826 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:00.258832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:00.258843 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:11:00.258881 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:00.258882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:00.258897 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:00.258977 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:00.259231 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.260331 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.261582 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.262240 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.262310 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.262364 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:00.262475 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 13:11:00.262518 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":100,"threads_started":1}
I20260925 13:11:00.262928 19379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.258542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:43223 (local address 127.0.0.1:43506)
0925 13:11:00.258969 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:00.258979 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:00.259047 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:00.259294 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:00.259295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:00.259297 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:00.259328 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:00.259329 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.260547 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.260548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:00.261434 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:00.261436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.262685 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.262686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:00.262765 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.262769 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:11:00.262823 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 13:11:00.262855 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":60,"threads_started":1}
I20260925 13:11:00.262959 19382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.258792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43506 (local address 127.0.206.65:43223)
0925 13:11:00.258978 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:00.258983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:00.259044 (+ 61us) server_negotiation.cc:407] Connection header received
0925 13:11:00.259119 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:00.259122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:00.259137 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:00.259210 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:00.259403 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.260442 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.261513 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.262232 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.262309 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.262363 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:00.262840 (+ 477us) server_negotiation.cc:299] Negotiation successful
0925 13:11:00.262881 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":70,"threads_started":1}
I20260925 13:11:00.263401 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.258837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33058 (local address 127.0.206.67:41597)
0925 13:11:00.258971 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:00.258975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:00.259164 (+ 189us) server_negotiation.cc:407] Connection header received
0925 13:11:00.259204 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:00.259206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:00.259221 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:00.259292 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:00.259486 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.260125 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.262398 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.263063 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.263138 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.263191 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:00.263286 (+ 95us) server_negotiation.cc:299] Negotiation successful
0925 13:11:00.263327 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
I20260925 13:11:00.263535 19170 tablet_service.cc:1467] Processing CreateTablet for tablet e2d114b8274941d0afc62a474d2d01ab (DEFAULT_TABLE table=test-table [id=ef14a233007d40ecadb1351d1ab411e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:00.263820 19036 tablet_service.cc:1467] Processing CreateTablet for tablet e2d114b8274941d0afc62a474d2d01ab (DEFAULT_TABLE table=test-table [id=ef14a233007d40ecadb1351d1ab411e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:00.263918 19170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2d114b8274941d0afc62a474d2d01ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:00.264106 19036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2d114b8274941d0afc62a474d2d01ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:00.266275 19384 tablet_bootstrap.cc:492] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be: Bootstrap starting.
I20260925 13:11:00.266723 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.258547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:41597 (local address 127.0.0.1:33058)
0925 13:11:00.259019 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:00.259028 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:00.259087 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:00.259365 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:00.259366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:00.259368 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:00.259410 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:00.259412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.260227 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.260228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:00.261503 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:00.261505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.263147 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.263148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:00.263210 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.263214 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:11:00.266565 (+ 3351us) client_negotiation.cc:240] Negotiation successful
0925 13:11:00.266619 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":4,"thread_start_us":61,"threads_started":1}
I20260925 13:11:00.266855 19385 tablet_bootstrap.cc:492] T e2d114b8274941d0afc62a474d2d01ab P 995080c25374464fb17d7ba0b4a11eb8: Bootstrap starting.
I20260925 13:11:00.267071 19384 tablet_bootstrap.cc:654] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:00.267426 19385 tablet_bootstrap.cc:654] T e2d114b8274941d0afc62a474d2d01ab P 995080c25374464fb17d7ba0b4a11eb8: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:00.267417 19304 tablet_service.cc:1467] Processing CreateTablet for tablet e2d114b8274941d0afc62a474d2d01ab (DEFAULT_TABLE table=test-table [id=ef14a233007d40ecadb1351d1ab411e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:00.267740 19304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2d114b8274941d0afc62a474d2d01ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:00.267860 19385 log.cc:824] T e2d114b8274941d0afc62a474d2d01ab P 995080c25374464fb17d7ba0b4a11eb8: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:00.268460 19385 tablet_bootstrap.cc:492] T e2d114b8274941d0afc62a474d2d01ab P 995080c25374464fb17d7ba0b4a11eb8: No bootstrap required, opened a new log
I20260925 13:11:00.268512 19385 ts_tablet_manager.cc:1397] T e2d114b8274941d0afc62a474d2d01ab P 995080c25374464fb17d7ba0b4a11eb8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:11:00.270004 19385 raft_consensus.cc:348] T e2d114b8274941d0afc62a474d2d01ab P 995080c25374464fb17d7ba0b4a11eb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de47e530bcc4ecea04dfe871cdf04be" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44717 } } peers { permanent_uuid: "995080c25374464fb17d7ba0b4a11eb8" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43223 } } peers { permanent_uuid: "83a900634fd74f958443b436097b9ffa" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41597 } }
I20260925 13:11:00.270148 19385 raft_consensus.cc:374] T e2d114b8274941d0afc62a474d2d01ab P 995080c25374464fb17d7ba0b4a11eb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:00.270170 19385 raft_consensus.cc:729] T e2d114b8274941d0afc62a474d2d01ab P 995080c25374464fb17d7ba0b4a11eb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 995080c25374464fb17d7ba0b4a11eb8, State: Initialized, Role: FOLLOWER
I20260925 13:11:00.270306 19385 consensus_queue.cc:260] T e2d114b8274941d0afc62a474d2d01ab P 995080c25374464fb17d7ba0b4a11eb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de47e530bcc4ecea04dfe871cdf04be" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44717 } } peers { permanent_uuid: "995080c25374464fb17d7ba0b4a11eb8" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43223 } } peers { permanent_uuid: "83a900634fd74f958443b436097b9ffa" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41597 } }
I20260925 13:11:00.270363 19387 tablet_bootstrap.cc:492] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa: Bootstrap starting.
I20260925 13:11:00.270750 19385 ts_tablet_manager.cc:1428] T e2d114b8274941d0afc62a474d2d01ab P 995080c25374464fb17d7ba0b4a11eb8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:11:00.270771 19102 heartbeater.cc:503] Master 127.0.206.126:42597 was elected leader, sending a full tablet report...
I20260925 13:11:00.270900 19384 log.cc:824] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:00.271010 19387 tablet_bootstrap.cc:654] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:00.271458 19387 log.cc:824] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:00.272199 19387 tablet_bootstrap.cc:492] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa: No bootstrap required, opened a new log
I20260925 13:11:00.272192 19384 tablet_bootstrap.cc:492] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be: No bootstrap required, opened a new log
I20260925 13:11:00.272261 19384 ts_tablet_manager.cc:1397] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260925 13:11:00.272261 19387 ts_tablet_manager.cc:1397] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 13:11:00.274178 19387 raft_consensus.cc:348] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de47e530bcc4ecea04dfe871cdf04be" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44717 } } peers { permanent_uuid: "995080c25374464fb17d7ba0b4a11eb8" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43223 } } peers { permanent_uuid: "83a900634fd74f958443b436097b9ffa" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41597 } }
I20260925 13:11:00.274202 19384 raft_consensus.cc:348] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de47e530bcc4ecea04dfe871cdf04be" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44717 } } peers { permanent_uuid: "995080c25374464fb17d7ba0b4a11eb8" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43223 } } peers { permanent_uuid: "83a900634fd74f958443b436097b9ffa" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41597 } }
I20260925 13:11:00.274349 19384 raft_consensus.cc:374] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:00.274349 19387 raft_consensus.cc:374] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:00.274385 19384 raft_consensus.cc:729] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1de47e530bcc4ecea04dfe871cdf04be, State: Initialized, Role: FOLLOWER
I20260925 13:11:00.274385 19387 raft_consensus.cc:729] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83a900634fd74f958443b436097b9ffa, State: Initialized, Role: FOLLOWER
I20260925 13:11:00.274498 19387 consensus_queue.cc:260] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de47e530bcc4ecea04dfe871cdf04be" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44717 } } peers { permanent_uuid: "995080c25374464fb17d7ba0b4a11eb8" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43223 } } peers { permanent_uuid: "83a900634fd74f958443b436097b9ffa" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41597 } }
I20260925 13:11:00.274498 19384 consensus_queue.cc:260] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de47e530bcc4ecea04dfe871cdf04be" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44717 } } peers { permanent_uuid: "995080c25374464fb17d7ba0b4a11eb8" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43223 } } peers { permanent_uuid: "83a900634fd74f958443b436097b9ffa" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41597 } }
I20260925 13:11:00.274783 19384 ts_tablet_manager.cc:1428] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260925 13:11:00.274837 19370 heartbeater.cc:503] Master 127.0.206.126:42597 was elected leader, sending a full tablet report...
I20260925 13:11:00.274801 19236 heartbeater.cc:503] Master 127.0.206.126:42597 was elected leader, sending a full tablet report...
I20260925 13:11:00.274783 19387 ts_tablet_manager.cc:1428] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260925 13:11:00.330873 19237 tablet.cc:2369] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:11:00.418982 19103 tablet.cc:2369] T e2d114b8274941d0afc62a474d2d01ab P 995080c25374464fb17d7ba0b4a11eb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:11:00.476691 19371 tablet.cc:2369] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:11:00.548475 19391 raft_consensus.cc:482] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:11:00.548607 19391 raft_consensus.cc:504] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de47e530bcc4ecea04dfe871cdf04be" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44717 } } peers { permanent_uuid: "995080c25374464fb17d7ba0b4a11eb8" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43223 } } peers { permanent_uuid: "83a900634fd74f958443b436097b9ffa" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41597 } }
I20260925 13:11:00.548967 19391 leader_election.cc:288] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1de47e530bcc4ecea04dfe871cdf04be (127.0.206.66:44717), 995080c25374464fb17d7ba0b4a11eb8 (127.0.206.65:43223)
I20260925 13:11:00.552232 19382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.549229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:41761 (local address 127.0.206.65:43223)
0925 13:11:00.549286 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:00.549289 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:00.549299 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:11:00.549330 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:00.549331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:00.549343 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:00.549408 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:00.549588 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.550202 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.551385 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.551896 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.551950 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.551987 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:00.552120 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 13:11:00.552164 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 13:11:00.552232 19372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.549153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:43223 (local address 127.0.206.67:41761)
0925 13:11:00.549206 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:00.549218 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:00.549261 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:00.549432 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:00.549433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:00.549434 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:00.549511 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:00.549512 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.550287 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.550288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:00.551300 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:00.551302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.551961 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.551962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:00.552035 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.552038 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:11:00.552113 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 13:11:00.552157 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":59}
I20260925 13:11:00.552459 19381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.549217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:48785 (local address 127.0.206.66:44717)
0925 13:11:00.549262 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:00.549264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:00.549406 (+ 142us) server_negotiation.cc:407] Connection header received
0925 13:11:00.549442 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:00.549442 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:00.549454 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:00.549513 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:00.549695 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.550357 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.551540 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.552109 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.552169 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.552176 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:00.552352 (+ 176us) server_negotiation.cc:299] Negotiation successful
0925 13:11:00.552390 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 13:11:00.552561 19056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2d114b8274941d0afc62a474d2d01ab" candidate_uuid: "83a900634fd74f958443b436097b9ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "995080c25374464fb17d7ba0b4a11eb8" is_pre_election: true
I20260925 13:11:00.552673 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.549174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:44717 (local address 127.0.206.67:48785)
0925 13:11:00.549334 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:00.549344 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:00.549399 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:00.549532 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:00.549533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:00.549534 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:00.549625 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:00.549626 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.550430 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.550430 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:00.551466 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:00.551468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.552183 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.552184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:00.552252 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.552257 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 13:11:00.552531 (+ 274us) client_negotiation.cc:240] Negotiation successful
0925 13:11:00.552578 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
I20260925 13:11:00.552727 19056 raft_consensus.cc:2454] T e2d114b8274941d0afc62a474d2d01ab P 995080c25374464fb17d7ba0b4a11eb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83a900634fd74f958443b436097b9ffa in term 0.
I20260925 13:11:00.552958 19259 leader_election.cc:302] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83a900634fd74f958443b436097b9ffa, 995080c25374464fb17d7ba0b4a11eb8; no voters:
I20260925 13:11:00.552979 19190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2d114b8274941d0afc62a474d2d01ab" candidate_uuid: "83a900634fd74f958443b436097b9ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1de47e530bcc4ecea04dfe871cdf04be" is_pre_election: true
I20260925 13:11:00.553097 19391 raft_consensus.cc:2781] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:11:00.553117 19190 raft_consensus.cc:2454] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83a900634fd74f958443b436097b9ffa in term 0.
I20260925 13:11:00.553146 19391 raft_consensus.cc:482] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:11:00.553176 19391 raft_consensus.cc:3037] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:00.553838 19391 raft_consensus.cc:504] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de47e530bcc4ecea04dfe871cdf04be" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44717 } } peers { permanent_uuid: "995080c25374464fb17d7ba0b4a11eb8" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43223 } } peers { permanent_uuid: "83a900634fd74f958443b436097b9ffa" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41597 } }
I20260925 13:11:00.554029 19391 leader_election.cc:288] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [CANDIDATE]: Term 1 election: Requested vote from peers 1de47e530bcc4ecea04dfe871cdf04be (127.0.206.66:44717), 995080c25374464fb17d7ba0b4a11eb8 (127.0.206.65:43223)
I20260925 13:11:00.554098 19190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2d114b8274941d0afc62a474d2d01ab" candidate_uuid: "83a900634fd74f958443b436097b9ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1de47e530bcc4ecea04dfe871cdf04be"
I20260925 13:11:00.554163 19190 raft_consensus.cc:3037] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:00.554179 19056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2d114b8274941d0afc62a474d2d01ab" candidate_uuid: "83a900634fd74f958443b436097b9ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "995080c25374464fb17d7ba0b4a11eb8"
I20260925 13:11:00.554246 19056 raft_consensus.cc:3037] T e2d114b8274941d0afc62a474d2d01ab P 995080c25374464fb17d7ba0b4a11eb8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:00.554889 19190 raft_consensus.cc:2454] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83a900634fd74f958443b436097b9ffa in term 1.
I20260925 13:11:00.555014 19056 raft_consensus.cc:2454] T e2d114b8274941d0afc62a474d2d01ab P 995080c25374464fb17d7ba0b4a11eb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83a900634fd74f958443b436097b9ffa in term 1.
I20260925 13:11:00.555058 19258 leader_election.cc:302] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1de47e530bcc4ecea04dfe871cdf04be, 83a900634fd74f958443b436097b9ffa; no voters:
I20260925 13:11:00.555140 19391 raft_consensus.cc:2781] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:00.555291 19391 raft_consensus.cc:686] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [term 1 LEADER]: Becoming Leader. State: Replica: 83a900634fd74f958443b436097b9ffa, State: Running, Role: LEADER
I20260925 13:11:00.555383 19391 consensus_queue.cc:237] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de47e530bcc4ecea04dfe871cdf04be" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44717 } } peers { permanent_uuid: "995080c25374464fb17d7ba0b4a11eb8" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43223 } } peers { permanent_uuid: "83a900634fd74f958443b436097b9ffa" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41597 } }
I20260925 13:11:00.556223 18911 catalog_manager.cc:5495] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa reported cstate change: term changed from 0 to 1, leader changed from <none> to 83a900634fd74f958443b436097b9ffa (127.0.206.67). New cstate: current_term: 1 leader_uuid: "83a900634fd74f958443b436097b9ffa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1de47e530bcc4ecea04dfe871cdf04be" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "995080c25374464fb17d7ba0b4a11eb8" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83a900634fd74f958443b436097b9ffa" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41597 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:00.616076 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.613963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56058 (local address 127.0.206.67:41597)
0925 13:11:00.614016 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:00.614018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:00.614026 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:11:00.614059 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:00.614062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:00.614086 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:00.614147 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:00.614262 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.614735 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.615430 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.615608 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.615658 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.615777 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:00.615855 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:00.615921 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:00.615994 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 13:11:00.616002 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 13:11:00.617835 19190 raft_consensus.cc:1261] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be [term 1 FOLLOWER]: Refusing update from remote peer 83a900634fd74f958443b436097b9ffa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:00.617862 19056 raft_consensus.cc:1261] T e2d114b8274941d0afc62a474d2d01ab P 995080c25374464fb17d7ba0b4a11eb8 [term 1 FOLLOWER]: Refusing update from remote peer 83a900634fd74f958443b436097b9ffa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:00.618224 19391 consensus_queue.cc:1035] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [LEADER]: Connected to new peer: Peer: permanent_uuid: "995080c25374464fb17d7ba0b4a11eb8" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:00.618350 19394 consensus_queue.cc:1035] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1de47e530bcc4ecea04dfe871cdf04be" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 44717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:00.620452 19397 mvcc.cc:205] Tried to move back new op lower bound from 7333240261087842304 to 7333240260835614720. Current Snapshot: MvccSnapshot[applied={T|T < 7333240261087842304}]
I20260925 13:11:00.622505 19399 mvcc.cc:205] Tried to move back new op lower bound from 7333240261087842304 to 7333240260835614720. Current Snapshot: MvccSnapshot[applied={T|T < 7333240261087842304 or (T in {7333240261087842304})}]
I20260925 13:11:00.622944 18911 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:11:00.623018 18911 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:11:00.623719 18911 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 66d88665d00b4e87969f23cec74eb183: Sending DeleteTablet for 3 replicas of tablet e2d114b8274941d0afc62a474d2d01ab
I20260925 13:11:00.624032 19304 tablet_service.cc:1514] Processing DeleteTablet for tablet e2d114b8274941d0afc62a474d2d01ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:11:00 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:33058
I20260925 13:11:00.624064 19170 tablet_service.cc:1514] Processing DeleteTablet for tablet e2d114b8274941d0afc62a474d2d01ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:11:00 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:33776
I20260925 13:11:00.624122 19036 tablet_service.cc:1514] Processing DeleteTablet for tablet e2d114b8274941d0afc62a474d2d01ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:11:00 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:43506
I20260925 13:11:00.624310 19409 tablet_replica.cc:330] T e2d114b8274941d0afc62a474d2d01ab P 995080c25374464fb17d7ba0b4a11eb8: stopping tablet replica
I20260925 13:11:00.624341 19407 tablet_replica.cc:330] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa: stopping tablet replica
I20260925 13:11:00.624426 19407 raft_consensus.cc:2229] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:00.624468 19409 raft_consensus.cc:2229] T e2d114b8274941d0afc62a474d2d01ab P 995080c25374464fb17d7ba0b4a11eb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:00.624528 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 18973
I20260925 13:11:00.624607 19407 raft_consensus.cc:2258] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:00.625331 19407 ts_tablet_manager.cc:1905] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:11:00.627311 19407 ts_tablet_manager.cc:1918] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:11:00.627375 19407 log.cc:1196] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/e2d114b8274941d0afc62a474d2d01ab
I20260925 13:11:00.627683 19407 ts_tablet_manager.cc:1939] T e2d114b8274941d0afc62a474d2d01ab P 83a900634fd74f958443b436097b9ffa: Deleting consensus metadata
I20260925 13:11:00.628048 18895 catalog_manager.cc:4841] TS 83a900634fd74f958443b436097b9ffa (127.0.206.67:41597): tablet e2d114b8274941d0afc62a474d2d01ab (table test-table [id=ef14a233007d40ecadb1351d1ab411e9]) successfully deleted
I20260925 13:11:00.630903 19408 tablet_replica.cc:330] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be: stopping tablet replica
I20260925 13:11:00.631007 19408 raft_consensus.cc:2229] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:00.631106 19408 raft_consensus.cc:2258] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:00.631559 19408 ts_tablet_manager.cc:1905] T e2d114b8274941d0afc62a474d2d01ab P 1de47e530bcc4ecea04dfe871cdf04be: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 13:11:00.631619 18897 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:43223 (error 108)
I20260925 13:11:00.631891 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 19107
I20260925 13:11:00.634229 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.634105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:43223 (local address 127.0.0.1:56840)
0925 13:11:00.634157 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:00.634193 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:43223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 13:11:00.634348 18897 catalog_manager.cc:4584] TS 995080c25374464fb17d7ba0b4a11eb8 (127.0.206.65:43223): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e2d114b8274941d0afc62a474d2d01ab: Network error: Client connection negotiation failed: client connection to 127.0.206.65:43223: connect: Connection refused (error 111)
I20260925 13:11:00.637881 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 19241
I20260925 13:11:00.637923 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.637800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:44717 (local address 127.0.0.1:39376)
0925 13:11:00.637861 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:00.637893 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.66:44717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 13:11:00.642180 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 18879
2026-09-25T13:11:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2164 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-25T13:11:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:00Z Disabled control of system clock
I20260925 13:11:00.669272 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:35617
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:39657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:35617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 13:11:00.746829 19414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:00.747016 19414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:00.747037 19414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:00.748114 19414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:00.748149 19414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:00.748170 19414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:00.748183 19414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:00.748194 19414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:00.749428 19414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:39657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:35617
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:35617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:00.749598 19414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:00.749807 19414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:00.754487 19422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:00.754617 19420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:00.754740 19419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:00.754720 19414 server_base.cc:1034] running on GCE node
I20260925 13:11:00.754979 19414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:00.755280 19414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:00.756454 19414 hybrid_clock.cc:648] HybridClock initialized: now 1790341860756434 us; error 37 us; skew 500 ppm
I20260925 13:11:00.761312 19414 webserver.cc:460] Webserver started at http://127.0.206.126:44557/ using document root <none> and password file <none>
I20260925 13:11:00.761559 19414 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:00.761626 19414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:00.761783 19414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:00.762948 19414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "e9a18997d70f4bec85efe42098e341c3"
format_stamp: "Formatted at 2026-09-25 13:11:00 on dist-test-slave-chkf"
server_key: "5f8cda16603146551fe4c82f1a0f5832"
server_key_iv: "9e003b881b023577f608b21b62ba7eb1"
server_key_version: "encryptionkey@0"
I20260925 13:11:00.763288 19414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "e9a18997d70f4bec85efe42098e341c3"
format_stamp: "Formatted at 2026-09-25 13:11:00 on dist-test-slave-chkf"
server_key: "5f8cda16603146551fe4c82f1a0f5832"
server_key_iv: "9e003b881b023577f608b21b62ba7eb1"
server_key_version: "encryptionkey@0"
I20260925 13:11:00.764523 19414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:11:00.765256 19428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:00.765447 19414 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:11:00.765544 19414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "e9a18997d70f4bec85efe42098e341c3"
format_stamp: "Formatted at 2026-09-25 13:11:00 on dist-test-slave-chkf"
server_key: "5f8cda16603146551fe4c82f1a0f5832"
server_key_iv: "9e003b881b023577f608b21b62ba7eb1"
server_key_version: "encryptionkey@0"
I20260925 13:11:00.765625 19414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:00.782891 19414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:00.783146 19414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:00.783293 19414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:00.786990 19414 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:35617
I20260925 13:11:00.787029 19480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:35617 every 8 connection(s)
I20260925 13:11:00.787331 19414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:00.788285 19481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:00.790330 19481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3: Bootstrap starting.
I20260925 13:11:00.790928 19481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:00.791211 19481 log.cc:824] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:00.791821 19481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3: No bootstrap required, opened a new log
I20260925 13:11:00.793046 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 19414
I20260925 13:11:00.793123 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:00.793308 825 external_mini_cluster.cc:1428] Setting key 75a6f03c4a1b6c7f35cee20530257218
I20260925 13:11:00.793427 19481 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9a18997d70f4bec85efe42098e341c3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35617 } }
I20260925 13:11:00.793608 19481 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:00.793710 19481 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a18997d70f4bec85efe42098e341c3, State: Initialized, Role: FOLLOWER
I20260925 13:11:00.793875 19481 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9a18997d70f4bec85efe42098e341c3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35617 } }
I20260925 13:11:00.793984 19481 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:00.794028 19481 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:00.794071 19481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:00.794747 19481 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9a18997d70f4bec85efe42098e341c3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35617 } }
I20260925 13:11:00.794896 19481 leader_election.cc:302] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9a18997d70f4bec85efe42098e341c3; no voters:
I20260925 13:11:00.795058 19481 leader_election.cc:288] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:00.795110 19486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:00.795288 19486 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3 [term 1 LEADER]: Becoming Leader. State: Replica: e9a18997d70f4bec85efe42098e341c3, State: Running, Role: LEADER
I20260925 13:11:00.795310 19481 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:00.795421 19486 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9a18997d70f4bec85efe42098e341c3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35617 } }
I20260925 13:11:00.795797 19488 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9a18997d70f4bec85efe42098e341c3. Latest consensus state: current_term: 1 leader_uuid: "e9a18997d70f4bec85efe42098e341c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9a18997d70f4bec85efe42098e341c3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35617 } } }
I20260925 13:11:00.795881 19488 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:00.796096 19487 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9a18997d70f4bec85efe42098e341c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9a18997d70f4bec85efe42098e341c3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35617 } } }
I20260925 13:11:00.796232 19487 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:00.796356 19492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:00.796830 19492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:00.799343 19492 catalog_manager.cc:1347] Generated new cluster ID: 5c2514bdd82d48b8a8e0bb7c5cb04afc
I20260925 13:11:00.799388 19485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.793807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38290 (local address 127.0.206.126:35617)
0925 13:11:00.794014 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:00.794023 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:00.794056 (+ 33us) server_negotiation.cc:407] Connection header received
0925 13:11:00.794165 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:00.794172 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:00.794237 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:00.794369 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:00.794555 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.796273 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.797380 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.797941 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.798068 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.798669 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:00.798673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:00.798673 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:00.798901 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:00.798936 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:00.798950 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:00.799029 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:00.799119 (+ 90us) server_negotiation.cc:299] Negotiation successful
0925 13:11:00.799190 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260925 13:11:00.799396 19492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:00.806734 19492 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:00.807379 19492 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:00.821076 19492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3: Generated new TSK 0
I20260925 13:11:00.821231 19492 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:00.822794 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:35617
--builtin_ntp_servers=127.0.206.84:39657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 13:11:00.902913 19505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:00.903133 19505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:00.903169 19505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:00.904279 19505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:00.904340 19505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:00.904397 19505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:11:00.905720 19505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:39657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:35617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:00.905956 19505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:00.906229 19505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:00.911909 19511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:00.912026 19505 server_base.cc:1034] running on GCE node
W20260925 13:11:00.911927 19510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:00.911928 19513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:00.912405 19505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:00.912686 19505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:00.913856 19505 hybrid_clock.cc:648] HybridClock initialized: now 1790341860913841 us; error 25 us; skew 500 ppm
I20260925 13:11:00.915241 19505 webserver.cc:460] Webserver started at http://127.0.206.65:38875/ using document root <none> and password file <none>
I20260925 13:11:00.915491 19505 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:00.915547 19505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:00.915666 19505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:00.917145 19505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "17e4574a5aed474b862e4fe319bcdc05"
format_stamp: "Formatted at 2026-09-25 13:11:00 on dist-test-slave-chkf"
server_key: "5e4da9395155af8a1772ae61376c18fe"
server_key_iv: "d2c606bfaa5415451976a679ec5928a9"
server_key_version: "encryptionkey@0"
I20260925 13:11:00.917547 19505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "17e4574a5aed474b862e4fe319bcdc05"
format_stamp: "Formatted at 2026-09-25 13:11:00 on dist-test-slave-chkf"
server_key: "5e4da9395155af8a1772ae61376c18fe"
server_key_iv: "d2c606bfaa5415451976a679ec5928a9"
server_key_version: "encryptionkey@0"
I20260925 13:11:00.919090 19505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260925 13:11:00.919870 19519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:00.920060 19505 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:11:00.920135 19505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "17e4574a5aed474b862e4fe319bcdc05"
format_stamp: "Formatted at 2026-09-25 13:11:00 on dist-test-slave-chkf"
server_key: "5e4da9395155af8a1772ae61376c18fe"
server_key_iv: "d2c606bfaa5415451976a679ec5928a9"
server_key_version: "encryptionkey@0"
I20260925 13:11:00.920224 19505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:00.929543 19505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:00.929806 19505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:00.929949 19505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:00.930169 19505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:00.930465 19505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:00.930500 19505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:00.930518 19505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:00.930562 19505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:00.936100 19505 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:34737
I20260925 13:11:00.936196 19632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:34737 every 8 connection(s)
I20260925 13:11:00.936475 19505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:11:00.937170 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 19505
I20260925 13:11:00.937263 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:11:00.937445 825 external_mini_cluster.cc:1428] Setting key 746783137b7f85a03d58844b1d4632d4
I20260925 13:11:00.938349 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:35617
--builtin_ntp_servers=127.0.206.84:39657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 13:11:00.942062 19485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.937212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:47245 (local address 127.0.206.126:35617)
0925 13:11:00.937273 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:00.937275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:00.937545 (+ 270us) server_negotiation.cc:407] Connection header received
0925 13:11:00.938076 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:00.938078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:00.938090 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:00.938156 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:00.938922 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.939468 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.940549 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.940842 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.940958 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.941802 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:00.941804 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:00.941804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:00.941809 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:00.941822 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:00.941827 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:00.941879 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:00.941952 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 13:11:00.941986 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 13:11:00.942291 19635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:00.937188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:35617 (local address 127.0.206.65:47245)
0925 13:11:00.937373 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:00.937414 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:00.937968 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:00.938229 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:00.938233 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:00.938243 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:00.938828 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:00.938833 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.939570 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.939571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:00.940416 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:00.940418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:00.940971 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:00.940972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:00.941116 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:00.941651 (+ 535us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:11:00.941651 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:00.941677 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:11:00.941698 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:00.941884 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:00.941889 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:00.941892 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:11:00.941956 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 13:11:00.942059 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260925 13:11:00.942921 19633 heartbeater.cc:347] Connected to a master server at 127.0.206.126:35617
I20260925 13:11:00.943029 19633 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:00.943203 19633 heartbeater.cc:511] Master 127.0.206.126:35617 requested a full tablet report, sending...
I20260925 13:11:00.943743 19445 ts_manager.cc:194] Registered new tserver with Master: 17e4574a5aed474b862e4fe319bcdc05 (127.0.206.65:34737)
I20260925 13:11:00.944703 19445 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.65:47245
W20260925 13:11:01.016836 19636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:01.017022 19636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:01.017043 19636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:01.018678 19636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:01.018757 19636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:01.018796 19636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:11:01.020062 19636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:39657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:35617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:01.020232 19636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:01.020449 19636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:01.025255 19644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:01.025246 19642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:01.025249 19641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:01.025646 19636 server_base.cc:1034] running on GCE node
I20260925 13:11:01.025856 19636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:01.026108 19636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:01.027261 19636 hybrid_clock.cc:648] HybridClock initialized: now 1790341861027236 us; error 41 us; skew 500 ppm
I20260925 13:11:01.028447 19636 webserver.cc:460] Webserver started at http://127.0.206.66:46833/ using document root <none> and password file <none>
I20260925 13:11:01.028685 19636 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:01.028754 19636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:01.028883 19636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:01.029906 19636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "ad3d37f92dfc4c9db4f9140ef051832e"
format_stamp: "Formatted at 2026-09-25 13:11:01 on dist-test-slave-chkf"
server_key: "188f5c4aee16a9e7147ad2509cfe1ae0"
server_key_iv: "5786bd4c8ce0c4b83f509e2d308e0d0c"
server_key_version: "encryptionkey@0"
I20260925 13:11:01.030258 19636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "ad3d37f92dfc4c9db4f9140ef051832e"
format_stamp: "Formatted at 2026-09-25 13:11:01 on dist-test-slave-chkf"
server_key: "188f5c4aee16a9e7147ad2509cfe1ae0"
server_key_iv: "5786bd4c8ce0c4b83f509e2d308e0d0c"
server_key_version: "encryptionkey@0"
I20260925 13:11:01.031584 19636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:11:01.032248 19650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:01.032409 19636 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:11:01.032526 19636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "ad3d37f92dfc4c9db4f9140ef051832e"
format_stamp: "Formatted at 2026-09-25 13:11:01 on dist-test-slave-chkf"
server_key: "188f5c4aee16a9e7147ad2509cfe1ae0"
server_key_iv: "5786bd4c8ce0c4b83f509e2d308e0d0c"
server_key_version: "encryptionkey@0"
I20260925 13:11:01.032603 19636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:01.040354 19636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:01.040623 19636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:01.040761 19636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:01.040957 19636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:01.041239 19636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:01.041291 19636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:01.041332 19636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:01.041360 19636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:01.047554 19636 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:45419
I20260925 13:11:01.047592 19763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:45419 every 8 connection(s)
I20260925 13:11:01.047884 19636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:11:01.052886 19485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:01.048571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:57119 (local address 127.0.206.126:35617)
0925 13:11:01.048635 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:01.048637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:01.048886 (+ 249us) server_negotiation.cc:407] Connection header received
0925 13:11:01.049395 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:01.049396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:01.049412 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:01.049467 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:01.050111 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.050599 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.051634 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.051824 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.051909 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:01.052618 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:01.052619 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:01.052620 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:01.052625 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:01.052638 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:01.052642 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:01.052693 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:01.052784 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 13:11:01.052812 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 13:11:01.052898 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 19636
I20260925 13:11:01.053001 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:11:01.053085 19766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:01.048550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:35617 (local address 127.0.206.66:57119)
0925 13:11:01.048743 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:01.048798 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:01.049286 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:01.049542 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:01.049546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:01.049555 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:01.050002 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:01.050007 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.050689 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.050690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:01.051531 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:01.051534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.051930 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.051930 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:01.052045 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:01.052488 (+ 443us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:11:01.052489 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:01.052514 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:11:01.052533 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:01.052700 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:01.052705 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:01.052708 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:11:01.052784 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 13:11:01.052872 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260925 13:11:01.053206 825 external_mini_cluster.cc:1428] Setting key 32a57660c43c83cd3e50f87ab6d430ca
I20260925 13:11:01.053668 19764 heartbeater.cc:347] Connected to a master server at 127.0.206.126:35617
I20260925 13:11:01.053751 19764 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:01.053905 19764 heartbeater.cc:511] Master 127.0.206.126:35617 requested a full tablet report, sending...
I20260925 13:11:01.054250 19444 ts_manager.cc:194] Registered new tserver with Master: ad3d37f92dfc4c9db4f9140ef051832e (127.0.206.66:45419)
I20260925 13:11:01.054275 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:35617
--builtin_ntp_servers=127.0.206.84:39657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 13:11:01.054651 19444 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.66:57119
W20260925 13:11:01.132150 19767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:01.132350 19767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:01.132376 19767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:01.133543 19767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:01.133584 19767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:01.133608 19767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:11:01.135020 19767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:39657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:35617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:01.135309 19767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:01.135602 19767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:01.140750 19773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:01.140760 19775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:01.140766 19772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:01.141073 19767 server_base.cc:1034] running on GCE node
I20260925 13:11:01.141305 19767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:01.141553 19767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:01.142778 19767 hybrid_clock.cc:648] HybridClock initialized: now 1790341861142761 us; error 25 us; skew 500 ppm
I20260925 13:11:01.144181 19767 webserver.cc:460] Webserver started at http://127.0.206.67:43441/ using document root <none> and password file <none>
I20260925 13:11:01.144430 19767 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:01.144502 19767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:01.144665 19767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:01.145720 19767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "a01aea3d25054c6390ac8162a286802d"
format_stamp: "Formatted at 2026-09-25 13:11:01 on dist-test-slave-chkf"
server_key: "c865b154ad017d393d5d884323fc3a93"
server_key_iv: "49cee01d91e1f397be31df0a52267a06"
server_key_version: "encryptionkey@0"
I20260925 13:11:01.146080 19767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "a01aea3d25054c6390ac8162a286802d"
format_stamp: "Formatted at 2026-09-25 13:11:01 on dist-test-slave-chkf"
server_key: "c865b154ad017d393d5d884323fc3a93"
server_key_iv: "49cee01d91e1f397be31df0a52267a06"
server_key_version: "encryptionkey@0"
I20260925 13:11:01.147547 19767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:11:01.148211 19781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:01.148378 19767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:11:01.148473 19767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "a01aea3d25054c6390ac8162a286802d"
format_stamp: "Formatted at 2026-09-25 13:11:01 on dist-test-slave-chkf"
server_key: "c865b154ad017d393d5d884323fc3a93"
server_key_iv: "49cee01d91e1f397be31df0a52267a06"
server_key_version: "encryptionkey@0"
I20260925 13:11:01.148569 19767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:01.166754 19767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:01.167021 19767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:01.167164 19767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:01.167368 19767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:01.167723 19767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:01.167786 19767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:01.167821 19767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:01.167848 19767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:01.173687 19767 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:33051
I20260925 13:11:01.173739 19894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:33051 every 8 connection(s)
I20260925 13:11:01.174119 19767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:11:01.178637 19485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:01.174736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:32893 (local address 127.0.206.126:35617)
0925 13:11:01.174799 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:01.174802 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:01.175087 (+ 285us) server_negotiation.cc:407] Connection header received
0925 13:11:01.175518 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:01.175519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:01.175530 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:01.175581 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:01.176172 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.176630 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.177521 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.177703 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.177757 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:01.178411 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:01.178412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:01.178412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:01.178416 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:01.178427 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:01.178431 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:01.178473 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:01.178560 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 13:11:01.178582 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 13:11:01.178915 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 19767
I20260925 13:11:01.179035 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:11:01.179211 825 external_mini_cluster.cc:1428] Setting key e24f9b7e872b57131777a26909d610b9
I20260925 13:11:01.179464 19897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:01.174688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:35617 (local address 127.0.206.67:32893)
0925 13:11:01.174926 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:01.174981 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:01.175435 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:01.175632 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:01.175635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:01.175643 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:01.176105 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:01.176109 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.176700 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.176701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:01.177442 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:01.177444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.177774 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.177775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:01.177873 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:01.178292 (+ 419us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:11:01.178293 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:01.178317 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:11:01.178337 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:01.178483 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:01.178487 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:01.178489 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:11:01.178537 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 13:11:01.178619 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":93,"threads_started":1}
I20260925 13:11:01.179903 19895 heartbeater.cc:347] Connected to a master server at 127.0.206.126:35617
I20260925 13:11:01.180007 19895 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:01.180186 19895 heartbeater.cc:511] Master 127.0.206.126:35617 requested a full tablet report, sending...
I20260925 13:11:01.180534 19444 ts_manager.cc:194] Registered new tserver with Master: a01aea3d25054c6390ac8162a286802d (127.0.206.67:33051)
I20260925 13:11:01.180912 19444 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.67:32893
I20260925 13:11:01.191066 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:11:01.194996 19485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:01.192336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38300 (local address 127.0.206.126:35617)
0925 13:11:01.192406 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:01.192408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:01.192528 (+ 120us) server_negotiation.cc:407] Connection header received
0925 13:11:01.192562 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:01.192563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:01.192580 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:01.192623 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:01.192744 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.193164 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.193850 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.194014 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.194060 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:01.194695 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:01.194696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:01.194696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:01.194699 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:01.194733 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:01.194736 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:01.194794 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:01.194895 (+ 101us) server_negotiation.cc:299] Negotiation successful
0925 13:11:01.194906 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 13:11:01.196609 19444 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:11:01.197995 19444 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:11:01.203883 19904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:01.200054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:45419 (local address 127.0.0.1:42962)
0925 13:11:01.200480 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:01.200488 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:01.200523 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:01.200799 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:01.200799 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:01.200801 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:01.200830 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:01.200832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.202067 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.202068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:01.202883 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:01.202886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.203622 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.203623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:01.203691 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:01.203703 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 13:11:01.203768 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 13:11:01.203824 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":76,"threads_started":1}
I20260925 13:11:01.203891 19908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:01.200462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42962 (local address 127.0.206.66:45419)
0925 13:11:01.200660 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:01.200663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:01.200673 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:11:01.200711 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:01.200713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:01.200728 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:01.200805 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:01.200893 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.201982 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.202978 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.203535 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.203623 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:01.203705 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:01.203777 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 13:11:01.203810 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":81,"threads_started":1}
I20260925 13:11:01.204190 19905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:01.200042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:34737 (local address 127.0.0.1:36576)
0925 13:11:01.200280 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:01.200319 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:01.200383 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:01.200656 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:01.200658 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:01.200660 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:01.200707 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:01.200709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.202051 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.202052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:01.203111 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:01.203113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.203942 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.203943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:01.203998 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:01.204002 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:11:01.204069 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 13:11:01.204112 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":93,"threads_started":1}
I20260925 13:11:01.204367 19903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:01.200010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:33051 (local address 127.0.0.1:44278)
0925 13:11:01.200280 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:01.200319 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:01.200383 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:01.200628 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:01.200630 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:01.200632 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:01.200690 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:01.200692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.201730 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.201731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:01.202946 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:01.202949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.204133 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.204135 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:01.204200 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:01.204204 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:11:01.204261 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 13:11:01.204297 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":49,"threads_started":1}
I20260925 13:11:01.204604 19906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:01.200068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44278 (local address 127.0.206.67:33051)
0925 13:11:01.200202 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:01.200206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:01.200463 (+ 257us) server_negotiation.cc:407] Connection header received
0925 13:11:01.200536 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:01.200539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:01.200555 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:01.200627 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:01.200754 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.201643 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.203043 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.203711 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.203783 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:01.203855 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:01.204471 (+ 616us) server_negotiation.cc:299] Negotiation successful
0925 13:11:01.204522 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
I20260925 13:11:01.204743 19698 tablet_service.cc:1467] Processing CreateTablet for tablet 15cf67e0e918485580f9c6a03e8b0a2f (DEFAULT_TABLE table=test-table [id=6a13b153f7dd4376ab9a7133f1d37717]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:01.205113 19698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15cf67e0e918485580f9c6a03e8b0a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:01.205235 19829 tablet_service.cc:1467] Processing CreateTablet for tablet 15cf67e0e918485580f9c6a03e8b0a2f (DEFAULT_TABLE table=test-table [id=6a13b153f7dd4376ab9a7133f1d37717]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:01.205533 19829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15cf67e0e918485580f9c6a03e8b0a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:01.208423 19909 tablet_bootstrap.cc:492] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e: Bootstrap starting.
I20260925 13:11:01.208640 19907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:01.200261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36576 (local address 127.0.206.65:34737)
0925 13:11:01.200423 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:01.200427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:01.200439 (+ 12us) server_negotiation.cc:407] Connection header received
0925 13:11:01.200478 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:01.200480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:01.200495 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:01.200580 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:01.200792 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.201960 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.203191 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.203880 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.203941 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:01.204010 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:01.204190 (+ 180us) server_negotiation.cc:299] Negotiation successful
0925 13:11:01.204234 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260925 13:11:01.208632 19910 tablet_bootstrap.cc:492] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d: Bootstrap starting.
I20260925 13:11:01.209081 19909 tablet_bootstrap.cc:654] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:01.209226 19910 tablet_bootstrap.cc:654] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:01.209182 19567 tablet_service.cc:1467] Processing CreateTablet for tablet 15cf67e0e918485580f9c6a03e8b0a2f (DEFAULT_TABLE table=test-table [id=6a13b153f7dd4376ab9a7133f1d37717]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:01.209410 19909 log.cc:824] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:01.209520 19910 log.cc:824] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:01.209533 19567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15cf67e0e918485580f9c6a03e8b0a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:01.210167 19910 tablet_bootstrap.cc:492] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d: No bootstrap required, opened a new log
I20260925 13:11:01.210218 19910 ts_tablet_manager.cc:1397] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:11:01.211373 19909 tablet_bootstrap.cc:492] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e: No bootstrap required, opened a new log
I20260925 13:11:01.211436 19909 ts_tablet_manager.cc:1397] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 13:11:01.212352 19913 tablet_bootstrap.cc:492] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05: Bootstrap starting.
I20260925 13:11:01.212394 19910 raft_consensus.cc:348] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01aea3d25054c6390ac8162a286802d" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33051 } } peers { permanent_uuid: "17e4574a5aed474b862e4fe319bcdc05" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34737 } } peers { permanent_uuid: "ad3d37f92dfc4c9db4f9140ef051832e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45419 } }
I20260925 13:11:01.212529 19910 raft_consensus.cc:374] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:01.212594 19910 raft_consensus.cc:729] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a01aea3d25054c6390ac8162a286802d, State: Initialized, Role: FOLLOWER
I20260925 13:11:01.212724 19910 consensus_queue.cc:260] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01aea3d25054c6390ac8162a286802d" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33051 } } peers { permanent_uuid: "17e4574a5aed474b862e4fe319bcdc05" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34737 } } peers { permanent_uuid: "ad3d37f92dfc4c9db4f9140ef051832e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45419 } }
I20260925 13:11:01.212961 19910 ts_tablet_manager.cc:1428] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 13:11:01.212987 19913 tablet_bootstrap.cc:654] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:01.213002 19895 heartbeater.cc:503] Master 127.0.206.126:35617 was elected leader, sending a full tablet report...
I20260925 13:11:01.213243 19913 log.cc:824] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:01.213907 19913 tablet_bootstrap.cc:492] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05: No bootstrap required, opened a new log
I20260925 13:11:01.213965 19913 ts_tablet_manager.cc:1397] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:11:01.213897 19909 raft_consensus.cc:348] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01aea3d25054c6390ac8162a286802d" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33051 } } peers { permanent_uuid: "17e4574a5aed474b862e4fe319bcdc05" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34737 } } peers { permanent_uuid: "ad3d37f92dfc4c9db4f9140ef051832e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45419 } }
I20260925 13:11:01.214047 19909 raft_consensus.cc:374] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:01.214082 19909 raft_consensus.cc:729] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad3d37f92dfc4c9db4f9140ef051832e, State: Initialized, Role: FOLLOWER
I20260925 13:11:01.214282 19909 consensus_queue.cc:260] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01aea3d25054c6390ac8162a286802d" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33051 } } peers { permanent_uuid: "17e4574a5aed474b862e4fe319bcdc05" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34737 } } peers { permanent_uuid: "ad3d37f92dfc4c9db4f9140ef051832e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45419 } }
I20260925 13:11:01.214533 19909 ts_tablet_manager.cc:1428] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 13:11:01.214586 19764 heartbeater.cc:503] Master 127.0.206.126:35617 was elected leader, sending a full tablet report...
I20260925 13:11:01.216315 19913 raft_consensus.cc:348] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01aea3d25054c6390ac8162a286802d" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33051 } } peers { permanent_uuid: "17e4574a5aed474b862e4fe319bcdc05" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34737 } } peers { permanent_uuid: "ad3d37f92dfc4c9db4f9140ef051832e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45419 } }
I20260925 13:11:01.216423 19913 raft_consensus.cc:374] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:01.216444 19913 raft_consensus.cc:729] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e4574a5aed474b862e4fe319bcdc05, State: Initialized, Role: FOLLOWER
I20260925 13:11:01.216539 19913 consensus_queue.cc:260] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01aea3d25054c6390ac8162a286802d" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33051 } } peers { permanent_uuid: "17e4574a5aed474b862e4fe319bcdc05" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34737 } } peers { permanent_uuid: "ad3d37f92dfc4c9db4f9140ef051832e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45419 } }
I20260925 13:11:01.216740 19913 ts_tablet_manager.cc:1428] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 13:11:01.216806 19633 heartbeater.cc:503] Master 127.0.206.126:35617 was elected leader, sending a full tablet report...
W20260925 13:11:01.298386 19765 tablet.cc:2369] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:11:01.424686 19896 tablet.cc:2369] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:11:01.437129 19634 tablet.cc:2369] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:11:01.511358 19917 raft_consensus.cc:482] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:11:01.511475 19917 raft_consensus.cc:504] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01aea3d25054c6390ac8162a286802d" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33051 } } peers { permanent_uuid: "17e4574a5aed474b862e4fe319bcdc05" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34737 } } peers { permanent_uuid: "ad3d37f92dfc4c9db4f9140ef051832e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45419 } }
I20260925 13:11:01.512090 19917 leader_election.cc:288] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a01aea3d25054c6390ac8162a286802d (127.0.206.67:33051), ad3d37f92dfc4c9db4f9140ef051832e (127.0.206.66:45419)
I20260925 13:11:01.514998 19918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:01.512285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:33051 (local address 127.0.206.65:59727)
0925 13:11:01.512407 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:01.512419 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:01.512484 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:01.512698 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:01.512699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:01.512701 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:01.512770 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:01.512772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.513426 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.513427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:01.514176 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:01.514178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.514736 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.514737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:01.514812 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:01.514816 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:11:01.514857 (+ 41us) client_negotiation.cc:240] Negotiation successful
0925 13:11:01.514907 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":57,"threads_started":1}
I20260925 13:11:01.514997 19906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:01.512364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:59727 (local address 127.0.206.67:33051)
0925 13:11:01.512414 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:01.512416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:01.512476 (+ 60us) server_negotiation.cc:407] Connection header received
0925 13:11:01.512584 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:01.512586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:01.512594 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:01.512675 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:01.512862 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.513312 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.514270 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.514632 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.514685 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:01.514690 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:01.514888 (+ 198us) server_negotiation.cc:299] Negotiation successful
0925 13:11:01.514928 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 13:11:01.515038 19908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:01.512411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:48723 (local address 127.0.206.66:45419)
0925 13:11:01.512455 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:01.512457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:01.512467 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:11:01.512594 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:01.512595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:01.512602 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:01.512646 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:01.512822 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.513299 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.514301 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.514643 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.514750 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:01.514757 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:01.514927 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 13:11:01.514968 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 13:11:01.515022 19919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:01.512286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:45419 (local address 127.0.206.65:48723)
0925 13:11:01.512418 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:01.512428 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:01.512483 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:01.512672 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:01.512673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:01.512675 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:01.512748 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:01.512751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.513377 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.513378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:01.514186 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:01.514188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.514753 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.514754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:01.514832 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:01.514836 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:11:01.514910 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 13:11:01.514944 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":4,"thread_start_us":57,"threads_started":1}
I20260925 13:11:01.515569 19849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15cf67e0e918485580f9c6a03e8b0a2f" candidate_uuid: "17e4574a5aed474b862e4fe319bcdc05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a01aea3d25054c6390ac8162a286802d" is_pre_election: true
I20260925 13:11:01.515625 19718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15cf67e0e918485580f9c6a03e8b0a2f" candidate_uuid: "17e4574a5aed474b862e4fe319bcdc05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad3d37f92dfc4c9db4f9140ef051832e" is_pre_election: true
I20260925 13:11:01.515751 19849 raft_consensus.cc:2454] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e4574a5aed474b862e4fe319bcdc05 in term 0.
I20260925 13:11:01.515772 19718 raft_consensus.cc:2454] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17e4574a5aed474b862e4fe319bcdc05 in term 0.
I20260925 13:11:01.515990 19522 leader_election.cc:302] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e4574a5aed474b862e4fe319bcdc05, a01aea3d25054c6390ac8162a286802d; no voters:
I20260925 13:11:01.516137 19917 raft_consensus.cc:2781] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:11:01.516192 19917 raft_consensus.cc:482] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:11:01.516207 19917 raft_consensus.cc:3037] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:01.516862 19917 raft_consensus.cc:504] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01aea3d25054c6390ac8162a286802d" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33051 } } peers { permanent_uuid: "17e4574a5aed474b862e4fe319bcdc05" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34737 } } peers { permanent_uuid: "ad3d37f92dfc4c9db4f9140ef051832e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45419 } }
I20260925 13:11:01.516997 19917 leader_election.cc:288] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [CANDIDATE]: Term 1 election: Requested vote from peers a01aea3d25054c6390ac8162a286802d (127.0.206.67:33051), ad3d37f92dfc4c9db4f9140ef051832e (127.0.206.66:45419)
I20260925 13:11:01.517115 19849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15cf67e0e918485580f9c6a03e8b0a2f" candidate_uuid: "17e4574a5aed474b862e4fe319bcdc05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a01aea3d25054c6390ac8162a286802d"
I20260925 13:11:01.517161 19718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15cf67e0e918485580f9c6a03e8b0a2f" candidate_uuid: "17e4574a5aed474b862e4fe319bcdc05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad3d37f92dfc4c9db4f9140ef051832e"
I20260925 13:11:01.517202 19849 raft_consensus.cc:3037] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:01.517263 19718 raft_consensus.cc:3037] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:01.517963 19849 raft_consensus.cc:2454] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e4574a5aed474b862e4fe319bcdc05 in term 1.
I20260925 13:11:01.518013 19718 raft_consensus.cc:2454] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17e4574a5aed474b862e4fe319bcdc05 in term 1.
I20260925 13:11:01.518157 19522 leader_election.cc:302] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17e4574a5aed474b862e4fe319bcdc05, a01aea3d25054c6390ac8162a286802d; no voters:
I20260925 13:11:01.518242 19917 raft_consensus.cc:2781] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:01.518373 19917 raft_consensus.cc:686] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [term 1 LEADER]: Becoming Leader. State: Replica: 17e4574a5aed474b862e4fe319bcdc05, State: Running, Role: LEADER
I20260925 13:11:01.518469 19917 consensus_queue.cc:237] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01aea3d25054c6390ac8162a286802d" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33051 } } peers { permanent_uuid: "17e4574a5aed474b862e4fe319bcdc05" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34737 } } peers { permanent_uuid: "ad3d37f92dfc4c9db4f9140ef051832e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45419 } }
I20260925 13:11:01.519290 19444 catalog_manager.cc:5495] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17e4574a5aed474b862e4fe319bcdc05 (127.0.206.65). New cstate: current_term: 1 leader_uuid: "17e4574a5aed474b862e4fe319bcdc05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a01aea3d25054c6390ac8162a286802d" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17e4574a5aed474b862e4fe319bcdc05" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad3d37f92dfc4c9db4f9140ef051832e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45419 } health_report { overall_health: UNKNOWN } } }
I20260925 13:11:01.554616 19907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:01.552529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36584 (local address 127.0.206.65:34737)
0925 13:11:01.552567 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:01.552569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:01.552592 (+ 23us) server_negotiation.cc:407] Connection header received
0925 13:11:01.552677 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:01.552677 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:01.552712 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:01.552769 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:01.552872 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.553344 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.554029 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:01.554204 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:01.554254 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:01.554373 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:01.554452 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:01.554536 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:01.554560 (+ 24us) server_negotiation.cc:299] Negotiation successful
0925 13:11:01.554568 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 13:11:01.556412 19849 raft_consensus.cc:1261] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d [term 1 FOLLOWER]: Refusing update from remote peer 17e4574a5aed474b862e4fe319bcdc05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:01.556452 19718 raft_consensus.cc:1261] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e [term 1 FOLLOWER]: Refusing update from remote peer 17e4574a5aed474b862e4fe319bcdc05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:01.556789 19920 consensus_queue.cc:1035] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad3d37f92dfc4c9db4f9140ef051832e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:01.556948 19917 consensus_queue.cc:1035] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a01aea3d25054c6390ac8162a286802d" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:01.558976 19923 mvcc.cc:205] Tried to move back new op lower bound from 7333240264932380672 to 7333240264780218368. Current Snapshot: MvccSnapshot[applied={T|T < 7333240264932380672}]
I20260925 13:11:01.559249 19924 mvcc.cc:205] Tried to move back new op lower bound from 7333240264932380672 to 7333240264780218368. Current Snapshot: MvccSnapshot[applied={T|T < 7333240264932380672}]
I20260925 13:11:01.560125 19925 mvcc.cc:205] Tried to move back new op lower bound from 7333240264932380672 to 7333240264780218368. Current Snapshot: MvccSnapshot[applied={T|T < 7333240264932380672}]
I20260925 13:11:01.561168 19444 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:11:01.561254 19444 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:11:01.562045 19444 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e9a18997d70f4bec85efe42098e341c3: Sending DeleteTablet for 3 replicas of tablet 15cf67e0e918485580f9c6a03e8b0a2f
I20260925 13:11:01.562285 19698 tablet_service.cc:1514] Processing DeleteTablet for tablet 15cf67e0e918485580f9c6a03e8b0a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:11:01 UTC) from {username='slave'} at 127.0.0.1:42962
I20260925 13:11:01.562337 19567 tablet_service.cc:1514] Processing DeleteTablet for tablet 15cf67e0e918485580f9c6a03e8b0a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:11:01 UTC) from {username='slave'} at 127.0.0.1:36576
I20260925 13:11:01.562369 19829 tablet_service.cc:1514] Processing DeleteTablet for tablet 15cf67e0e918485580f9c6a03e8b0a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:11:01 UTC) from {username='slave'} at 127.0.0.1:44278
I20260925 13:11:01.562526 19828 tablet_service.cc:1514] Processing DeleteTablet for tablet 15cf67e0e918485580f9c6a03e8b0a2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:11:01 UTC) from {username='slave'} at 127.0.0.1:44278
I20260925 13:11:01.562620 19934 tablet_replica.cc:330] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d: stopping tablet replica
I20260925 13:11:01.562669 19933 tablet_replica.cc:330] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05: stopping tablet replica
I20260925 13:11:01.562768 19934 raft_consensus.cc:2229] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:01.562774 19933 raft_consensus.cc:2229] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:01.562855 19933 raft_consensus.cc:2258] T 15cf67e0e918485580f9c6a03e8b0a2f P 17e4574a5aed474b862e4fe319bcdc05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:01.562893 19932 tablet_replica.cc:330] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e: stopping tablet replica
I20260925 13:11:01.562896 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 19505
I20260925 13:11:01.562928 19934 raft_consensus.cc:2258] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:01.563349 19934 ts_tablet_manager.cc:1905] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:11:01.563680 19932 raft_consensus.cc:2229] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:01.563797 19932 raft_consensus.cc:2258] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:01.564211 19932 ts_tablet_manager.cc:1905] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:11:01.564869 19934 ts_tablet_manager.cc:1918] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:11:01.564950 19934 log.cc:1196] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/15cf67e0e918485580f9c6a03e8b0a2f
I20260925 13:11:01.565191 19934 ts_tablet_manager.cc:1939] T 15cf67e0e918485580f9c6a03e8b0a2f P a01aea3d25054c6390ac8162a286802d: Deleting consensus metadata
I20260925 13:11:01.565562 19431 catalog_manager.cc:4841] TS a01aea3d25054c6390ac8162a286802d (127.0.206.67:33051): tablet 15cf67e0e918485580f9c6a03e8b0a2f (table test-table [id=6a13b153f7dd4376ab9a7133f1d37717]) successfully deleted
W20260925 13:11:01.565692 19431 catalog_manager.cc:4808] TS a01aea3d25054c6390ac8162a286802d (127.0.206.67:33051): delete failed for tablet 15cf67e0e918485580f9c6a03e8b0a2f because the tablet was not found. No further retry: Not found: Tablet not found: 15cf67e0e918485580f9c6a03e8b0a2f
I20260925 13:11:01.566035 19932 ts_tablet_manager.cc:1918] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:11:01.566107 19932 log.cc:1196] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/15cf67e0e918485580f9c6a03e8b0a2f
I20260925 13:11:01.566340 19932 ts_tablet_manager.cc:1939] T 15cf67e0e918485580f9c6a03e8b0a2f P ad3d37f92dfc4c9db4f9140ef051832e: Deleting consensus metadata
I20260925 13:11:01.567060 19429 catalog_manager.cc:4841] TS ad3d37f92dfc4c9db4f9140ef051832e (127.0.206.66:45419): tablet 15cf67e0e918485580f9c6a03e8b0a2f (table test-table [id=6a13b153f7dd4376ab9a7133f1d37717]) successfully deleted
W20260925 13:11:01.569945 19432 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:34737 (error 108)
I20260925 13:11:01.570042 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 19636
I20260925 13:11:01.570958 19903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:01.570840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:34737 (local address 127.0.0.1:36598)
0925 13:11:01.570895 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:01.570935 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:34737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 13:11:01.571087 19432 catalog_manager.cc:4584] TS 17e4574a5aed474b862e4fe319bcdc05 (127.0.206.65:34737): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 15cf67e0e918485580f9c6a03e8b0a2f: Network error: Client connection negotiation failed: client connection to 127.0.206.65:34737: connect: Connection refused (error 111)
I20260925 13:11:01.574987 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 19767
I20260925 13:11:01.579169 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 19414
2026-09-25T13:11:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (935 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:01 dist-test-slave-chkf krb5kdc[19938](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:01 dist-test-slave-chkf krb5kdc[19938](info): set up 2 sockets
Sep 25 13:11:01 dist-test-slave-chkf krb5kdc[19938](info): commencing operation
krb5kdc: starting...
W20260925 13:11:02.602504 825 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:02 dist-test-slave-chkf krb5kdc[19938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:11:02.711437 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:46273
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:46273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:02.788223 19951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:02.788395 19951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:02.788417 19951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:02.789444 19951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:02.789476 19951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:02.789490 19951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:02.789502 19951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:02.789515 19951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:02.790871 19951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:46273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:46273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:02.791030 19951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:02.791173 19951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:02.795692 19959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:02.795842 19956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:02.795889 19951 server_base.cc:1034] running on GCE node
W20260925 13:11:02.795902 19957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:02.796285 19951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:02.796554 19951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:02.797711 19951 hybrid_clock.cc:648] HybridClock initialized: now 1790341862797711 us; error 42 us; skew 500 ppm
Sep 25 13:11:02 dist-test-slave-chkf krb5kdc[19938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341862, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:02.799973 19951 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:11:02.800367 19951 webserver.cc:460] Webserver started at http://127.0.206.126:43545/ using document root <none> and password file <none>
I20260925 13:11:02.800594 19951 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:02.800661 19951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:02.800808 19951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:02.801761 19951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "4b66ed00fd6346dd98de87c01eec30c6"
format_stamp: "Formatted at 2026-09-25 13:11:02 on dist-test-slave-chkf"
server_key: "6ca080f662ec3caf285491444c1cb8cf"
server_key_iv: "69cf8e8e79541339bb4245e6d0ea77b2"
server_key_version: "encryptionkey@0"
I20260925 13:11:02.802119 19951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "4b66ed00fd6346dd98de87c01eec30c6"
format_stamp: "Formatted at 2026-09-25 13:11:02 on dist-test-slave-chkf"
server_key: "6ca080f662ec3caf285491444c1cb8cf"
server_key_iv: "69cf8e8e79541339bb4245e6d0ea77b2"
server_key_version: "encryptionkey@0"
I20260925 13:11:02.803431 19951 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:11:02.804086 19966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:02.804250 19951 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:11:02.804337 19951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "4b66ed00fd6346dd98de87c01eec30c6"
format_stamp: "Formatted at 2026-09-25 13:11:02 on dist-test-slave-chkf"
server_key: "6ca080f662ec3caf285491444c1cb8cf"
server_key_iv: "69cf8e8e79541339bb4245e6d0ea77b2"
server_key_version: "encryptionkey@0"
I20260925 13:11:02.804414 19951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:02.845629 19951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:02.847555 19951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:02.847716 19951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:02.851560 19951 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:46273
I20260925 13:11:02.851610 20018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:46273 every 8 connection(s)
I20260925 13:11:02.851979 19951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:02.852839 20019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:02.855042 20019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6: Bootstrap starting.
I20260925 13:11:02.855649 20019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:02.855700 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 19951
I20260925 13:11:02.855818 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:02.855975 20019 log.cc:824] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:02.856029 825 external_mini_cluster.cc:1428] Setting key 468aaadc48c61685027ebb6e663692e5
I20260925 13:11:02.856801 20019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6: No bootstrap required, opened a new log
I20260925 13:11:02.859311 20019 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b66ed00fd6346dd98de87c01eec30c6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46273 } }
I20260925 13:11:02.859479 20019 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:02.859513 20019 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b66ed00fd6346dd98de87c01eec30c6, State: Initialized, Role: FOLLOWER
I20260925 13:11:02.859633 20019 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b66ed00fd6346dd98de87c01eec30c6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46273 } }
I20260925 13:11:02.859704 20019 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:02.859730 20019 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:02.859774 20019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:02.860366 20019 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b66ed00fd6346dd98de87c01eec30c6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46273 } }
I20260925 13:11:02.860487 20019 leader_election.cc:302] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b66ed00fd6346dd98de87c01eec30c6; no voters:
I20260925 13:11:02.860675 20019 leader_election.cc:288] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:02.860695 20024 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:02.860818 20024 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6 [term 1 LEADER]: Becoming Leader. State: Replica: 4b66ed00fd6346dd98de87c01eec30c6, State: Running, Role: LEADER
I20260925 13:11:02.860927 20019 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:02.860920 20024 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b66ed00fd6346dd98de87c01eec30c6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46273 } }
I20260925 13:11:02.862346 20026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b66ed00fd6346dd98de87c01eec30c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b66ed00fd6346dd98de87c01eec30c6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46273 } } }
I20260925 13:11:02.862440 20026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:02.862633 20025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b66ed00fd6346dd98de87c01eec30c6. Latest consensus state: current_term: 1 leader_uuid: "4b66ed00fd6346dd98de87c01eec30c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b66ed00fd6346dd98de87c01eec30c6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46273 } } }
I20260925 13:11:02.862744 20025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:02.862646 20039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:02.863474 20039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 13:11:02 dist-test-slave-chkf krb5kdc[19938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:11:02.865182 20039 catalog_manager.cc:1347] Generated new cluster ID: 03554088ee084148a88e970629c32bcb
I20260925 13:11:02.865249 20039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:02.868644 20023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:02.856507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55654 (local address 127.0.206.126:46273)
0925 13:11:02.856681 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:02.856688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:02.856712 (+ 24us) server_negotiation.cc:407] Connection header received
0925 13:11:02.857160 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:02.857167 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:02.857213 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:02.857323 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:02.858096 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:02.859013 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:02.860045 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:02.860439 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:02.860562 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:02.864915 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:02.864920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:02.864920 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:02.864932 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:02.867072 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:02.867493 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:02.867496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:02.867496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:02.867590 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:02.867803 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:02.867803 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:02.867804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:02.868071 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:02.868220 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:02.868328 (+ 108us) server_negotiation.cc:299] Negotiation successful
0925 13:11:02.868447 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":98,"threads_started":1}
I20260925 13:11:02.878278 20039 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:02.878829 20039 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:02.882759 20039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6: Generated new TSK 0
I20260925 13:11:02.882931 20039 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260925 13:11:02.909873 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:46273
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:02.991294 20045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:02.991480 20045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:02.991503 20045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:02.992547 20045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:02.992580 20045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:02.992605 20045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:11:02.994061 20045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:46273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:02.994238 20045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:02.994390 20045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:02.999030 20053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:02.999038 20050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:02.999017 20051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:02.999078 20045 server_base.cc:1034] running on GCE node
I20260925 13:11:02.999500 20045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:02.999768 20045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:03.000911 20045 hybrid_clock.cc:648] HybridClock initialized: now 1790341863000898 us; error 31 us; skew 500 ppm
Sep 25 13:11:03 dist-test-slave-chkf krb5kdc[19938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341863, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:03.003304 20045 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260925 13:11:03.003739 20045 webserver.cc:460] Webserver started at http://127.0.206.65:40271/ using document root <none> and password file <none>
I20260925 13:11:03.003947 20045 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:03.003984 20045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:03.004065 20045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:03.004981 20045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "1c23764dc9954db899a295550cab034d"
format_stamp: "Formatted at 2026-09-25 13:11:03 on dist-test-slave-chkf"
server_key: "f8fc342cf053b8996e28c40f5ddec060"
server_key_iv: "325a597f5c89408b33d48a10e58d1a59"
server_key_version: "encryptionkey@0"
I20260925 13:11:03.005259 20045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "1c23764dc9954db899a295550cab034d"
format_stamp: "Formatted at 2026-09-25 13:11:03 on dist-test-slave-chkf"
server_key: "f8fc342cf053b8996e28c40f5ddec060"
server_key_iv: "325a597f5c89408b33d48a10e58d1a59"
server_key_version: "encryptionkey@0"
I20260925 13:11:03.006417 20045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 13:11:03.007138 20060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:03.007302 20045 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:03.007396 20045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "1c23764dc9954db899a295550cab034d"
format_stamp: "Formatted at 2026-09-25 13:11:03 on dist-test-slave-chkf"
server_key: "f8fc342cf053b8996e28c40f5ddec060"
server_key_iv: "325a597f5c89408b33d48a10e58d1a59"
server_key_version: "encryptionkey@0"
I20260925 13:11:03.007485 20045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:03.014881 20045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:03.016497 20045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:03.016652 20045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:03.016846 20045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:03.017113 20045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:03.017169 20045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:03.017208 20045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:03.017236 20045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:03.022765 20045 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:42105
I20260925 13:11:03.022799 20173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:42105 every 8 connection(s)
I20260925 13:11:03.023080 20045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:11:03.023953 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 20045
I20260925 13:11:03.024101 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:11:03.024305 825 external_mini_cluster.cc:1428] Setting key d2d61e06da7992b34402ee2577f4ea4a
Sep 25 13:11:03 dist-test-slave-chkf krb5kdc[19938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341863, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
I20260925 13:11:03.032487 20023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.023740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:41909 (local address 127.0.206.126:46273)
0925 13:11:03.023816 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:03.023818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:03.024010 (+ 192us) server_negotiation.cc:407] Connection header received
0925 13:11:03.025141 (+ 1131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:03.025142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:03.025154 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:03.025209 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:03.025791 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.026291 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.027342 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.027586 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.027663 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.029678 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:03.029679 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:03.029680 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:03.029687 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:03.031114 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:03.031508 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:03.031509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:03.031510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:03.031568 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:03.031818 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:03.031819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:03.031819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:03.031964 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:03.032042 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:03.032302 (+ 260us) server_negotiation.cc:299] Negotiation successful
0925 13:11:03.032396 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 13:11:03.032651 20176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.023737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:46273 (local address 127.0.206.65:41909)
0925 13:11:03.023902 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:03.023934 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:03.025045 (+ 1111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:03.025275 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:03.025279 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:03.025521 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:03.025698 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:03.025702 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.026384 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.026385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:03.027255 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:03.027257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.027681 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.027682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:03.027805 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.028293 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:03.028300 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:03.029565 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:03.031210 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:03.031217 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:03.031219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:03.031434 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:03.031637 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:03.031638 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:03.031638 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:03.031749 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:03.032033 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:03.032039 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:03.032176 (+ 137us) client_negotiation.cc:769] Sending connection context
0925 13:11:03.032282 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 13:11:03.032424 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260925 13:11:03.033146 20174 heartbeater.cc:347] Connected to a master server at 127.0.206.126:46273
I20260925 13:11:03.033236 20174 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:03.033399 20174 heartbeater.cc:511] Master 127.0.206.126:46273 requested a full tablet report, sending...
I20260925 13:11:03.033931 19983 ts_manager.cc:194] Registered new tserver with Master: 1c23764dc9954db899a295550cab034d (127.0.206.65:42105)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
I20260925 13:11:03.034761 19983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:41909
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260925 13:11:03.048609 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:46273
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:03.128093 20179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:03.128331 20179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:03.128376 20179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:03.129518 20179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:03.129590 20179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:03.129638 20179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:11:03.131124 20179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:46273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:03.131373 20179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:03.131596 20179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 13:11:03.136513 20179 server_base.cc:1034] running on GCE node
W20260925 13:11:03.136554 20184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:03.136561 20187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:03.136557 20185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:03.136967 20179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:03.137166 20179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:03.138329 20179 hybrid_clock.cc:648] HybridClock initialized: now 1790341863138318 us; error 41 us; skew 500 ppm
Sep 25 13:11:03 dist-test-slave-chkf krb5kdc[19938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341863, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:03.140784 20179 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260925 13:11:03.141209 20179 webserver.cc:460] Webserver started at http://127.0.206.66:34065/ using document root <none> and password file <none>
I20260925 13:11:03.141379 20179 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:03.141417 20179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:03.141501 20179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:03.142441 20179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "bbd857c4e7994e03a110a32440885575"
format_stamp: "Formatted at 2026-09-25 13:11:03 on dist-test-slave-chkf"
server_key: "a53124090181cb80c5798acba2c9bea7"
server_key_iv: "4efa8f516af10a1673a08caa3872b439"
server_key_version: "encryptionkey@0"
I20260925 13:11:03.142822 20179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "bbd857c4e7994e03a110a32440885575"
format_stamp: "Formatted at 2026-09-25 13:11:03 on dist-test-slave-chkf"
server_key: "a53124090181cb80c5798acba2c9bea7"
server_key_iv: "4efa8f516af10a1673a08caa3872b439"
server_key_version: "encryptionkey@0"
I20260925 13:11:03.144039 20179 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:11:03.144690 20194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:03.144842 20179 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:03.144922 20179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "bbd857c4e7994e03a110a32440885575"
format_stamp: "Formatted at 2026-09-25 13:11:03 on dist-test-slave-chkf"
server_key: "a53124090181cb80c5798acba2c9bea7"
server_key_iv: "4efa8f516af10a1673a08caa3872b439"
server_key_version: "encryptionkey@0"
I20260925 13:11:03.144973 20179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:03.154421 20179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:03.156596 20179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:03.156697 20179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:03.156857 20179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:03.157088 20179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:03.157119 20179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:03.157140 20179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:03.157152 20179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:03.162825 20179 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:37187
I20260925 13:11:03.162914 20307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:37187 every 8 connection(s)
I20260925 13:11:03.163188 20179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
Sep 25 13:11:03 dist-test-slave-chkf krb5kdc[19938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341863, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:11:03.171417 20023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.163902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:41031 (local address 127.0.206.126:46273)
0925 13:11:03.163964 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:03.163965 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:03.164163 (+ 198us) server_negotiation.cc:407] Connection header received
0925 13:11:03.164984 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:03.164985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:03.165009 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:03.165072 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:03.165598 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.166099 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.167089 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.167284 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.167345 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.169181 (+ 1836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:03.169182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:03.169182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:03.169188 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:03.170328 (+ 1140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:03.170663 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:03.170663 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:03.170664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:03.170745 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:03.170942 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:03.170943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:03.170943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:03.171035 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:03.171085 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:03.171298 (+ 213us) server_negotiation.cc:299] Negotiation successful
0925 13:11:03.171357 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 13:11:03.171547 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.163877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:46273 (local address 127.0.206.66:41031)
0925 13:11:03.164015 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:03.164052 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:03.164897 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:03.165127 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:03.165130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:03.165351 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:03.165521 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:03.165524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.166175 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.166176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:03.167016 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:03.167018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.167358 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.167359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:03.167446 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.167830 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:03.167838 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:03.169103 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:03.170395 (+ 1292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:03.170400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:03.170402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:03.170600 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:03.170798 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:03.170798 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:03.170799 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:03.170887 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:03.171095 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:03.171099 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:03.171196 (+ 97us) client_negotiation.cc:769] Sending connection context
0925 13:11:03.171272 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 13:11:03.171375 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260925 13:11:03.172005 20308 heartbeater.cc:347] Connected to a master server at 127.0.206.126:46273
I20260925 13:11:03.172104 20308 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:03.172288 20308 heartbeater.cc:511] Master 127.0.206.126:46273 requested a full tablet report, sending...
I20260925 13:11:03.172650 19983 ts_manager.cc:194] Registered new tserver with Master: bbd857c4e7994e03a110a32440885575 (127.0.206.66:37187)
I20260925 13:11:03.172895 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 20179
I20260925 13:11:03.173002 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:11:03.173062 19983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:41031
I20260925 13:11:03.173194 825 external_mini_cluster.cc:1428] Setting key 8f1b0e232babe1aaef53a0e188e3948d
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260925 13:11:03.195540 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:46273
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:03.273414 20313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:03.273602 20313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:03.273625 20313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:03.274747 20313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:03.274796 20313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:03.274844 20313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:11:03.276242 20313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:46273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:03.276453 20313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:03.276667 20313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:03.281381 20319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:03.281409 20318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:03.281488 20313 server_base.cc:1034] running on GCE node
W20260925 13:11:03.281375 20321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:03.281814 20313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:03.281996 20313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:03.283154 20313 hybrid_clock.cc:648] HybridClock initialized: now 1790341863283140 us; error 43 us; skew 500 ppm
Sep 25 13:11:03 dist-test-slave-chkf krb5kdc[19938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341863, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:03.285391 20313 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260925 13:11:03.285786 20313 webserver.cc:460] Webserver started at http://127.0.206.67:41109/ using document root <none> and password file <none>
I20260925 13:11:03.285995 20313 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:03.286057 20313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:03.286208 20313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:03.287211 20313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "ae0eb2887dab474fbe4223222520ad24"
format_stamp: "Formatted at 2026-09-25 13:11:03 on dist-test-slave-chkf"
server_key: "9715b4f1f71fd484cd531d0f68652c0d"
server_key_iv: "c3c6075adcdad5597561d1045f12e6a8"
server_key_version: "encryptionkey@0"
I20260925 13:11:03.287556 20313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "ae0eb2887dab474fbe4223222520ad24"
format_stamp: "Formatted at 2026-09-25 13:11:03 on dist-test-slave-chkf"
server_key: "9715b4f1f71fd484cd531d0f68652c0d"
server_key_iv: "c3c6075adcdad5597561d1045f12e6a8"
server_key_version: "encryptionkey@0"
I20260925 13:11:03.288784 20313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:11:03.289510 20328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:03.289677 20313 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:11:03.289769 20313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "ae0eb2887dab474fbe4223222520ad24"
format_stamp: "Formatted at 2026-09-25 13:11:03 on dist-test-slave-chkf"
server_key: "9715b4f1f71fd484cd531d0f68652c0d"
server_key_iv: "c3c6075adcdad5597561d1045f12e6a8"
server_key_version: "encryptionkey@0"
I20260925 13:11:03.289848 20313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:03.302414 20313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:03.304302 20313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:03.304448 20313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:03.304656 20313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:03.304940 20313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:03.304994 20313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:03.305034 20313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:03.305063 20313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:03.311187 20313 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:43761
I20260925 13:11:03.311249 20441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:43761 every 8 connection(s)
I20260925 13:11:03.311530 20313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
Sep 25 13:11:03 dist-test-slave-chkf krb5kdc[19938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341863, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:11:03.319602 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 20313
I20260925 13:11:03.319705 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:11:03.319907 825 external_mini_cluster.cc:1428] Setting key bd3f9edbdd35feaee7793725424f0627
I20260925 13:11:03.320438 20023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.312244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:56101 (local address 127.0.206.126:46273)
0925 13:11:03.312310 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:03.312312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:03.312507 (+ 195us) server_negotiation.cc:407] Connection header received
0925 13:11:03.313352 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:03.313353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:03.313362 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:03.313411 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:03.313945 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.314431 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.315433 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.315615 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.315668 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.317606 (+ 1938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:03.317607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:03.317607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:03.317614 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:03.319085 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:03.319435 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:03.319436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:03.319436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:03.319499 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:03.319754 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:03.319756 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:03.319756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:03.319877 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:03.319956 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:03.320261 (+ 305us) server_negotiation.cc:299] Negotiation successful
0925 13:11:03.320339 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 13:11:03.320686 20444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.312215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:46273 (local address 127.0.206.67:56101)
0925 13:11:03.312382 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:03.312422 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:03.313270 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:03.313471 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:03.313474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:03.313710 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:03.313879 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:03.313882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.314506 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.314506 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:03.315361 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:03.315363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.315703 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.315705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:03.315811 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.316236 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:03.316243 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:03.317529 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:03.319145 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:03.319149 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:03.319151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:03.319354 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:03.319565 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:03.319566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:03.319566 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:03.319689 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:03.319947 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:03.319953 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:03.320091 (+ 138us) client_negotiation.cc:769] Sending connection context
0925 13:11:03.320218 (+ 127us) client_negotiation.cc:240] Negotiation successful
0925 13:11:03.320396 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
I20260925 13:11:03.321216 20442 heartbeater.cc:347] Connected to a master server at 127.0.206.126:46273
I20260925 13:11:03.321318 20442 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:03.321480 20442 heartbeater.cc:511] Master 127.0.206.126:46273 requested a full tablet report, sending...
I20260925 13:11:03.321848 19983 ts_manager.cc:194] Registered new tserver with Master: ae0eb2887dab474fbe4223222520ad24 (127.0.206.67:43761)
I20260925 13:11:03.322165 19983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:56101
I20260925 13:11:03.331588 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:11:03.338651 20023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.332877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55656 (local address 127.0.206.126:46273)
0925 13:11:03.332947 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:03.332949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:03.333059 (+ 110us) server_negotiation.cc:407] Connection header received
0925 13:11:03.333142 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:03.333143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:03.333152 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:03.333220 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:03.333828 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.334283 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.335226 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.335402 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.335455 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.336489 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:03.336490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:03.336490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:03.336496 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:03.337812 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:03.338096 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:03.338096 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:03.338097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:03.338142 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:03.338279 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:03.338280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:03.338280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:03.338390 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:03.338457 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:03.338537 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 13:11:03.338594 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 13:11:03.340145 19983 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55656:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:11:03.341673 19983 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:11:03.347836 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.343757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:37187 (local address 127.0.0.1:55564)
0925 13:11:03.344005 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:03.344014 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:03.344077 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:03.344458 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:03.344461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:03.344463 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:03.344526 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:03.344527 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.345408 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.345409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:03.346557 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:03.346559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.347560 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.347562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:03.347654 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.347659 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 13:11:03.347699 (+ 40us) client_negotiation.cc:240] Negotiation successful
0925 13:11:03.347760 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":61,"threads_started":1}
I20260925 13:11:03.347939 20451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.343727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:42105 (local address 127.0.0.1:45004)
0925 13:11:03.343958 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:03.343996 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:03.344077 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:03.344472 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:03.344473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:03.344475 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:03.344534 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:03.344536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.345822 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.345823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:03.346781 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:03.346784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.347558 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.347559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:03.347638 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.347651 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 13:11:03.347785 (+ 134us) client_negotiation.cc:240] Negotiation successful
0925 13:11:03.347826 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":2,"thread_start_us":60,"threads_started":1}
I20260925 13:11:03.348062 20452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.343757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:43761 (local address 127.0.0.1:33452)
0925 13:11:03.344170 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:03.344178 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:03.344243 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:03.344500 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:03.344501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:03.344502 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:03.344535 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:03.344537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.345794 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.345795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:03.346907 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:03.346909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.347845 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.347846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:03.347913 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.347917 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 13:11:03.347963 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 13:11:03.348000 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":34,"thread_start_us":64,"threads_started":1}
I20260925 13:11:03.348168 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.344119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55564 (local address 127.0.206.66:37187)
0925 13:11:03.344290 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:03.344294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:03.344303 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:11:03.344336 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:03.344338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:03.344353 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:03.344598 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:03.344674 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.345307 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.346657 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.347447 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.347957 (+ 510us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.348016 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:03.348040 (+ 24us) server_negotiation.cc:299] Negotiation successful
0925 13:11:03.348082 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260925 13:11:03.348315 20454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.343942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33452 (local address 127.0.206.67:43761)
0925 13:11:03.344112 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:03.344117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:03.344295 (+ 178us) server_negotiation.cc:407] Connection header received
0925 13:11:03.344336 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:03.344338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:03.344353 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:03.344424 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:03.344627 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.345687 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.346983 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.347700 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.347760 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.347812 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:03.348208 (+ 396us) server_negotiation.cc:299] Negotiation successful
0925 13:11:03.348245 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260925 13:11:03.348336 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.343836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45004 (local address 127.0.206.65:42105)
0925 13:11:03.344237 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:03.344242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:03.344251 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:11:03.344290 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:03.344292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:03.344305 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:03.344380 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:03.344592 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.345683 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.346860 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.347468 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.347546 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.347615 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:03.347802 (+ 187us) server_negotiation.cc:299] Negotiation successful
0925 13:11:03.347844 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":84,"threads_started":1}
I20260925 13:11:03.348914 20242 tablet_service.cc:1467] Processing CreateTablet for tablet b9a64f4e208e4911a9e3e736bcddc08c (DEFAULT_TABLE table=test-table [id=8b4787b9c3ac4a23a7060ac64d2b6244]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:03.348932 20108 tablet_service.cc:1467] Processing CreateTablet for tablet b9a64f4e208e4911a9e3e736bcddc08c (DEFAULT_TABLE table=test-table [id=8b4787b9c3ac4a23a7060ac64d2b6244]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:03.348973 20376 tablet_service.cc:1467] Processing CreateTablet for tablet b9a64f4e208e4911a9e3e736bcddc08c (DEFAULT_TABLE table=test-table [id=8b4787b9c3ac4a23a7060ac64d2b6244]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:03.349256 20242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9a64f4e208e4911a9e3e736bcddc08c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:03.349260 20108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9a64f4e208e4911a9e3e736bcddc08c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:03.349260 20376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9a64f4e208e4911a9e3e736bcddc08c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:03.352480 20458 tablet_bootstrap.cc:492] T b9a64f4e208e4911a9e3e736bcddc08c P 1c23764dc9954db899a295550cab034d: Bootstrap starting.
I20260925 13:11:03.352555 20456 tablet_bootstrap.cc:492] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575: Bootstrap starting.
I20260925 13:11:03.353050 20458 tablet_bootstrap.cc:654] T b9a64f4e208e4911a9e3e736bcddc08c P 1c23764dc9954db899a295550cab034d: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:03.353071 20456 tablet_bootstrap.cc:654] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:03.353521 20458 log.cc:824] T b9a64f4e208e4911a9e3e736bcddc08c P 1c23764dc9954db899a295550cab034d: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:03.353539 20456 log.cc:824] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:03.353856 20457 tablet_bootstrap.cc:492] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24: Bootstrap starting.
I20260925 13:11:03.354257 20456 tablet_bootstrap.cc:492] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575: No bootstrap required, opened a new log
I20260925 13:11:03.354337 20456 ts_tablet_manager.cc:1397] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:11:03.354383 20457 tablet_bootstrap.cc:654] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:03.354929 20457 log.cc:824] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:03.355098 20458 tablet_bootstrap.cc:492] T b9a64f4e208e4911a9e3e736bcddc08c P 1c23764dc9954db899a295550cab034d: No bootstrap required, opened a new log
I20260925 13:11:03.355154 20458 ts_tablet_manager.cc:1397] T b9a64f4e208e4911a9e3e736bcddc08c P 1c23764dc9954db899a295550cab034d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 13:11:03.355700 20457 tablet_bootstrap.cc:492] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24: No bootstrap required, opened a new log
I20260925 13:11:03.355765 20457 ts_tablet_manager.cc:1397] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 13:11:03.356420 20456 raft_consensus.cc:348] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c23764dc9954db899a295550cab034d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42105 } } peers { permanent_uuid: "bbd857c4e7994e03a110a32440885575" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37187 } } peers { permanent_uuid: "ae0eb2887dab474fbe4223222520ad24" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43761 } }
I20260925 13:11:03.356585 20456 raft_consensus.cc:374] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:03.356630 20456 raft_consensus.cc:729] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbd857c4e7994e03a110a32440885575, State: Initialized, Role: FOLLOWER
I20260925 13:11:03.356632 20458 raft_consensus.cc:348] T b9a64f4e208e4911a9e3e736bcddc08c P 1c23764dc9954db899a295550cab034d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c23764dc9954db899a295550cab034d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42105 } } peers { permanent_uuid: "bbd857c4e7994e03a110a32440885575" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37187 } } peers { permanent_uuid: "ae0eb2887dab474fbe4223222520ad24" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43761 } }
I20260925 13:11:03.356755 20456 consensus_queue.cc:260] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c23764dc9954db899a295550cab034d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42105 } } peers { permanent_uuid: "bbd857c4e7994e03a110a32440885575" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37187 } } peers { permanent_uuid: "ae0eb2887dab474fbe4223222520ad24" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43761 } }
I20260925 13:11:03.356827 20458 raft_consensus.cc:374] T b9a64f4e208e4911a9e3e736bcddc08c P 1c23764dc9954db899a295550cab034d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:03.356873 20458 raft_consensus.cc:729] T b9a64f4e208e4911a9e3e736bcddc08c P 1c23764dc9954db899a295550cab034d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c23764dc9954db899a295550cab034d, State: Initialized, Role: FOLLOWER
I20260925 13:11:03.357014 20456 ts_tablet_manager.cc:1428] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 13:11:03.356985 20458 consensus_queue.cc:260] T b9a64f4e208e4911a9e3e736bcddc08c P 1c23764dc9954db899a295550cab034d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c23764dc9954db899a295550cab034d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42105 } } peers { permanent_uuid: "bbd857c4e7994e03a110a32440885575" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37187 } } peers { permanent_uuid: "ae0eb2887dab474fbe4223222520ad24" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43761 } }
I20260925 13:11:03.357051 20308 heartbeater.cc:503] Master 127.0.206.126:46273 was elected leader, sending a full tablet report...
I20260925 13:11:03.357239 20458 ts_tablet_manager.cc:1428] T b9a64f4e208e4911a9e3e736bcddc08c P 1c23764dc9954db899a295550cab034d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:11:03.357331 20174 heartbeater.cc:503] Master 127.0.206.126:46273 was elected leader, sending a full tablet report...
I20260925 13:11:03.357688 20457 raft_consensus.cc:348] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c23764dc9954db899a295550cab034d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42105 } } peers { permanent_uuid: "bbd857c4e7994e03a110a32440885575" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37187 } } peers { permanent_uuid: "ae0eb2887dab474fbe4223222520ad24" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43761 } }
I20260925 13:11:03.357849 20457 raft_consensus.cc:374] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:03.357883 20457 raft_consensus.cc:729] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0eb2887dab474fbe4223222520ad24, State: Initialized, Role: FOLLOWER
I20260925 13:11:03.357990 20457 consensus_queue.cc:260] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c23764dc9954db899a295550cab034d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42105 } } peers { permanent_uuid: "bbd857c4e7994e03a110a32440885575" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37187 } } peers { permanent_uuid: "ae0eb2887dab474fbe4223222520ad24" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43761 } }
I20260925 13:11:03.358239 20457 ts_tablet_manager.cc:1428] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 13:11:03.358268 20442 heartbeater.cc:503] Master 127.0.206.126:46273 was elected leader, sending a full tablet report...
W20260925 13:11:03.413743 20309 tablet.cc:2369] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:11:03.523738 20175 tablet.cc:2369] T b9a64f4e208e4911a9e3e736bcddc08c P 1c23764dc9954db899a295550cab034d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:11:03.562055 20443 tablet.cc:2369] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:11:03.595041 20462 raft_consensus.cc:482] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:11:03.595211 20462 raft_consensus.cc:504] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c23764dc9954db899a295550cab034d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42105 } } peers { permanent_uuid: "bbd857c4e7994e03a110a32440885575" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37187 } } peers { permanent_uuid: "ae0eb2887dab474fbe4223222520ad24" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43761 } }
I20260925 13:11:03.595538 20462 leader_election.cc:288] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c23764dc9954db899a295550cab034d (127.0.206.65:42105), ae0eb2887dab474fbe4223222520ad24 (127.0.206.67:43761)
I20260925 13:11:03.598410 20454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.595866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:35881 (local address 127.0.206.67:43761)
0925 13:11:03.595926 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:03.595927 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:03.595936 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:11:03.595962 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:03.595963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:03.595975 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:03.596039 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:03.596206 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.596692 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.597640 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.598051 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.598107 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.598136 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:03.598299 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 13:11:03.598342 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 13:11:03.598419 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.595708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:43761 (local address 127.0.206.66:35881)
0925 13:11:03.595791 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:03.595804 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:03.595863 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:03.596049 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:03.596050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:03.596052 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:03.596134 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:03.596136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.596773 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.596773 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:03.597561 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:03.597562 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.598145 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.598146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:03.598223 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.598226 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:11:03.598278 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 13:11:03.598333 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 13:11:03.598456 20453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.595798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:53089 (local address 127.0.206.65:42105)
0925 13:11:03.595882 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:03.595884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:03.595952 (+ 68us) server_negotiation.cc:407] Connection header received
0925 13:11:03.596018 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:03.596019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:03.596032 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:03.596084 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:03.596285 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.596756 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.597723 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.598061 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.598151 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.598159 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:03.598349 (+ 190us) server_negotiation.cc:299] Negotiation successful
0925 13:11:03.598388 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 13:11:03.598443 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.595707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:42105 (local address 127.0.206.66:53089)
0925 13:11:03.595894 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:03.595903 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:03.595962 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:03.596106 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:03.596107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:03.596109 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:03.596199 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:03.596201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.596833 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.596834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:03.597637 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:03.597639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.598167 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.598168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:03.598250 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.598253 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:11:03.598327 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 13:11:03.598360 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":30,"thread_start_us":94,"threads_started":1}
I20260925 13:11:03.598814 20396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9a64f4e208e4911a9e3e736bcddc08c" candidate_uuid: "bbd857c4e7994e03a110a32440885575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0eb2887dab474fbe4223222520ad24" is_pre_election: true
I20260925 13:11:03.598963 20396 raft_consensus.cc:2454] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbd857c4e7994e03a110a32440885575 in term 0.
I20260925 13:11:03.599041 20128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9a64f4e208e4911a9e3e736bcddc08c" candidate_uuid: "bbd857c4e7994e03a110a32440885575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c23764dc9954db899a295550cab034d" is_pre_election: true
I20260925 13:11:03.599171 20128 raft_consensus.cc:2454] T b9a64f4e208e4911a9e3e736bcddc08c P 1c23764dc9954db899a295550cab034d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbd857c4e7994e03a110a32440885575 in term 0.
I20260925 13:11:03.599395 20197 leader_election.cc:302] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c23764dc9954db899a295550cab034d, bbd857c4e7994e03a110a32440885575; no voters:
I20260925 13:11:03.599530 20462 raft_consensus.cc:2781] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:11:03.599596 20462 raft_consensus.cc:482] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:11:03.599625 20462 raft_consensus.cc:3037] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:03.600238 20462 raft_consensus.cc:504] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c23764dc9954db899a295550cab034d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42105 } } peers { permanent_uuid: "bbd857c4e7994e03a110a32440885575" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37187 } } peers { permanent_uuid: "ae0eb2887dab474fbe4223222520ad24" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43761 } }
I20260925 13:11:03.600364 20462 leader_election.cc:288] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [CANDIDATE]: Term 1 election: Requested vote from peers 1c23764dc9954db899a295550cab034d (127.0.206.65:42105), ae0eb2887dab474fbe4223222520ad24 (127.0.206.67:43761)
I20260925 13:11:03.600520 20396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9a64f4e208e4911a9e3e736bcddc08c" candidate_uuid: "bbd857c4e7994e03a110a32440885575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0eb2887dab474fbe4223222520ad24"
I20260925 13:11:03.600548 20128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9a64f4e208e4911a9e3e736bcddc08c" candidate_uuid: "bbd857c4e7994e03a110a32440885575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c23764dc9954db899a295550cab034d"
I20260925 13:11:03.600590 20396 raft_consensus.cc:3037] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:03.600688 20128 raft_consensus.cc:3037] T b9a64f4e208e4911a9e3e736bcddc08c P 1c23764dc9954db899a295550cab034d [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:03.601356 20396 raft_consensus.cc:2454] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbd857c4e7994e03a110a32440885575 in term 1.
I20260925 13:11:03.601502 20128 raft_consensus.cc:2454] T b9a64f4e208e4911a9e3e736bcddc08c P 1c23764dc9954db899a295550cab034d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbd857c4e7994e03a110a32440885575 in term 1.
I20260925 13:11:03.601512 20195 leader_election.cc:302] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae0eb2887dab474fbe4223222520ad24, bbd857c4e7994e03a110a32440885575; no voters:
I20260925 13:11:03.601648 20462 raft_consensus.cc:2781] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:03.601781 20462 raft_consensus.cc:686] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [term 1 LEADER]: Becoming Leader. State: Replica: bbd857c4e7994e03a110a32440885575, State: Running, Role: LEADER
I20260925 13:11:03.601869 20462 consensus_queue.cc:237] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c23764dc9954db899a295550cab034d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42105 } } peers { permanent_uuid: "bbd857c4e7994e03a110a32440885575" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37187 } } peers { permanent_uuid: "ae0eb2887dab474fbe4223222520ad24" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43761 } }
I20260925 13:11:03.602591 19982 catalog_manager.cc:5495] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 reported cstate change: term changed from 0 to 1, leader changed from <none> to bbd857c4e7994e03a110a32440885575 (127.0.206.66). New cstate: current_term: 1 leader_uuid: "bbd857c4e7994e03a110a32440885575" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c23764dc9954db899a295550cab034d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbd857c4e7994e03a110a32440885575" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae0eb2887dab474fbe4223222520ad24" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43761 } health_report { overall_health: UNKNOWN } } }
I20260925 13:11:03.627161 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.625048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55570 (local address 127.0.206.66:37187)
0925 13:11:03.625107 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:03.625109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:03.625130 (+ 21us) server_negotiation.cc:407] Connection header received
0925 13:11:03.625157 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:03.625158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:03.625172 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:03.625227 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:03.625350 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.625758 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.626452 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.626618 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.626666 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.626859 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:03.626940 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:03.626997 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:03.627036 (+ 39us) server_negotiation.cc:299] Negotiation successful
0925 13:11:03.627052 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 13:11:03.628803 20128 raft_consensus.cc:1261] T b9a64f4e208e4911a9e3e736bcddc08c P 1c23764dc9954db899a295550cab034d [term 1 FOLLOWER]: Refusing update from remote peer bbd857c4e7994e03a110a32440885575: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:03.628805 20396 raft_consensus.cc:1261] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24 [term 1 FOLLOWER]: Refusing update from remote peer bbd857c4e7994e03a110a32440885575: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:03.629155 20466 consensus_queue.cc:1035] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0eb2887dab474fbe4223222520ad24" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:03.629276 20462 consensus_queue.cc:1035] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c23764dc9954db899a295550cab034d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:03.631811 20471 mvcc.cc:205] Tried to move back new op lower bound from 7333240273420832768 to 7333240273313681408. Current Snapshot: MvccSnapshot[applied={T|T < 7333240273420832768}]
I20260925 13:11:03.631825 20470 mvcc.cc:205] Tried to move back new op lower bound from 7333240273420832768 to 7333240273313681408. Current Snapshot: MvccSnapshot[applied={T|T < 7333240273420832768}]
I20260925 13:11:03.633967 19982 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:11:03.634053 19982 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:11:03.635938 19982 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b66ed00fd6346dd98de87c01eec30c6: Sending DeleteTablet for 3 replicas of tablet b9a64f4e208e4911a9e3e736bcddc08c
I20260925 13:11:03.636268 20242 tablet_service.cc:1514] Processing DeleteTablet for tablet b9a64f4e208e4911a9e3e736bcddc08c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:11:03 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:55564
I20260925 13:11:03.636492 20376 tablet_service.cc:1514] Processing DeleteTablet for tablet b9a64f4e208e4911a9e3e736bcddc08c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:11:03 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:33452
I20260925 13:11:03.636548 20479 tablet_replica.cc:330] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575: stopping tablet replica
I20260925 13:11:03.636636 20479 raft_consensus.cc:2229] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:03.636749 20479 raft_consensus.cc:2258] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:03.636765 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 20045
I20260925 13:11:03.636869 20108 tablet_service.cc:1514] Processing DeleteTablet for tablet b9a64f4e208e4911a9e3e736bcddc08c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:11:03 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:45004
I20260925 13:11:03.637356 20480 tablet_replica.cc:330] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24: stopping tablet replica
I20260925 13:11:03.637437 20480 raft_consensus.cc:2229] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:03.637524 20480 raft_consensus.cc:2258] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:03.637934 20480 ts_tablet_manager.cc:1905] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:11:03.640012 20480 ts_tablet_manager.cc:1918] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:11:03.640090 20480 log.cc:1196] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/b9a64f4e208e4911a9e3e736bcddc08c
I20260925 13:11:03.640188 20479 ts_tablet_manager.cc:1905] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:11:03.640369 20480 ts_tablet_manager.cc:1939] T b9a64f4e208e4911a9e3e736bcddc08c P ae0eb2887dab474fbe4223222520ad24: Deleting consensus metadata
I20260925 13:11:03.641014 19967 catalog_manager.cc:4841] TS ae0eb2887dab474fbe4223222520ad24 (127.0.206.67:43761): tablet b9a64f4e208e4911a9e3e736bcddc08c (table test-table [id=8b4787b9c3ac4a23a7060ac64d2b6244]) successfully deleted
I20260925 13:11:03.641610 20479 ts_tablet_manager.cc:1918] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:11:03.641693 20479 log.cc:1196] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/b9a64f4e208e4911a9e3e736bcddc08c
I20260925 13:11:03.641925 20479 ts_tablet_manager.cc:1939] T b9a64f4e208e4911a9e3e736bcddc08c P bbd857c4e7994e03a110a32440885575: Deleting consensus metadata
I20260925 13:11:03.642464 19970 catalog_manager.cc:4841] TS bbd857c4e7994e03a110a32440885575 (127.0.206.66:37187): tablet b9a64f4e208e4911a9e3e736bcddc08c (table test-table [id=8b4787b9c3ac4a23a7060ac64d2b6244]) successfully deleted
W20260925 13:11:03.642879 19969 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:42105 (error 108)
I20260925 13:11:03.643112 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 20179
I20260925 13:11:03.643311 20452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.643197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:42105 (local address 127.0.0.1:45010)
0925 13:11:03.643244 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:03.643275 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:42105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 13:11:03.643402 19969 catalog_manager.cc:4584] TS 1c23764dc9954db899a295550cab034d (127.0.206.65:42105): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b9a64f4e208e4911a9e3e736bcddc08c: Network error: Client connection negotiation failed: client connection to 127.0.206.65:42105: connect: Connection refused (error 111)
I20260925 13:11:03.648268 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 20313
I20260925 13:11:03.652868 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 19951
2026-09-25T13:11:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2074 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-25T13:11:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:03Z Disabled control of system clock
I20260925 13:11:03.679471 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:44353
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:32825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:44353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 13:11:03.758035 20485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:03.758224 20485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:03.758244 20485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:03.759389 20485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:03.759426 20485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:03.759439 20485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:03.759451 20485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:03.759500 20485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:03.760782 20485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:32825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:44353
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:44353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:03.761019 20485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:03.761296 20485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:03.765848 20490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:03.765882 20491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:03.765882 20493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:03.766073 20485 server_base.cc:1034] running on GCE node
I20260925 13:11:03.766386 20485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:03.766675 20485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:03.767848 20485 hybrid_clock.cc:648] HybridClock initialized: now 1790341863767828 us; error 35 us; skew 500 ppm
I20260925 13:11:03.769058 20485 webserver.cc:460] Webserver started at http://127.0.206.126:40063/ using document root <none> and password file <none>
I20260925 13:11:03.769297 20485 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:03.769367 20485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:03.769511 20485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:03.770470 20485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "bec179aa1bfc4b4db0f061fb776972d0"
format_stamp: "Formatted at 2026-09-25 13:11:03 on dist-test-slave-chkf"
server_key: "19135853941106327b1a632bae1088b0"
server_key_iv: "8b922a7b3f2a62b6c427909685227edb"
server_key_version: "encryptionkey@0"
I20260925 13:11:03.770850 20485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "bec179aa1bfc4b4db0f061fb776972d0"
format_stamp: "Formatted at 2026-09-25 13:11:03 on dist-test-slave-chkf"
server_key: "19135853941106327b1a632bae1088b0"
server_key_iv: "8b922a7b3f2a62b6c427909685227edb"
server_key_version: "encryptionkey@0"
I20260925 13:11:03.772078 20485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:11:03.772778 20499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:03.772948 20485 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:11:03.773056 20485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "bec179aa1bfc4b4db0f061fb776972d0"
format_stamp: "Formatted at 2026-09-25 13:11:03 on dist-test-slave-chkf"
server_key: "19135853941106327b1a632bae1088b0"
server_key_iv: "8b922a7b3f2a62b6c427909685227edb"
server_key_version: "encryptionkey@0"
I20260925 13:11:03.773140 20485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:03.791437 20485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:03.791704 20485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:03.791850 20485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:03.795545 20485 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:44353
I20260925 13:11:03.795600 20551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:44353 every 8 connection(s)
I20260925 13:11:03.795917 20485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:03.796774 20552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:03.798805 20552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0: Bootstrap starting.
I20260925 13:11:03.799289 20552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:03.799513 20552 log.cc:824] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:03.800066 20552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0: No bootstrap required, opened a new log
I20260925 13:11:03.801517 20552 raft_consensus.cc:348] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec179aa1bfc4b4db0f061fb776972d0" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44353 } }
I20260925 13:11:03.801638 20552 raft_consensus.cc:374] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:03.801661 20552 raft_consensus.cc:729] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bec179aa1bfc4b4db0f061fb776972d0, State: Initialized, Role: FOLLOWER
I20260925 13:11:03.801786 20552 consensus_queue.cc:260] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec179aa1bfc4b4db0f061fb776972d0" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44353 } }
I20260925 13:11:03.801865 20552 raft_consensus.cc:388] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:03.801893 20552 raft_consensus.cc:482] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:03.801925 20552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:03.802372 20552 raft_consensus.cc:504] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec179aa1bfc4b4db0f061fb776972d0" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44353 } }
I20260925 13:11:03.802454 20552 leader_election.cc:302] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bec179aa1bfc4b4db0f061fb776972d0; no voters:
I20260925 13:11:03.802609 20552 leader_election.cc:288] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:03.802649 20555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:03.802793 20555 raft_consensus.cc:686] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0 [term 1 LEADER]: Becoming Leader. State: Replica: bec179aa1bfc4b4db0f061fb776972d0, State: Running, Role: LEADER
I20260925 13:11:03.802887 20552 sys_catalog.cc:564] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:03.802897 20555 consensus_queue.cc:237] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec179aa1bfc4b4db0f061fb776972d0" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44353 } }
I20260925 13:11:03.803328 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 20485
I20260925 13:11:03.803450 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:03.803705 825 external_mini_cluster.cc:1428] Setting key 33397279be3b2c1851304901843aa29a
I20260925 13:11:03.804148 20556 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bec179aa1bfc4b4db0f061fb776972d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec179aa1bfc4b4db0f061fb776972d0" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44353 } } }
I20260925 13:11:03.804246 20556 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:03.804419 20557 sys_catalog.cc:455] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bec179aa1bfc4b4db0f061fb776972d0. Latest consensus state: current_term: 1 leader_uuid: "bec179aa1bfc4b4db0f061fb776972d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bec179aa1bfc4b4db0f061fb776972d0" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44353 } } }
I20260925 13:11:03.804461 20571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:03.804495 20557 sys_catalog.cc:458] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:03.805009 20571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:03.806475 20571 catalog_manager.cc:1347] Generated new cluster ID: 6a0b13bdc26846b186747e3be8969a33
I20260925 13:11:03.806527 20571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:03.809376 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.804359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47510 (local address 127.0.206.126:44353)
0925 13:11:03.804564 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:03.804573 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:03.804795 (+ 222us) server_negotiation.cc:407] Connection header received
0925 13:11:03.804914 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:03.804921 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:03.804984 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:03.805139 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:03.805364 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.806694 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.807493 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.807902 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.808043 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.808730 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:03.808733 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:03.808734 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:03.808902 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:03.808933 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:03.808942 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:03.809013 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:03.809094 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 13:11:03.809177 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":76,"threads_started":1}
I20260925 13:11:03.826282 20571 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:03.826876 20571 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:03.837060 20571 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0: Generated new TSK 0
I20260925 13:11:03.837247 20571 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:03.841769 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:44353
--builtin_ntp_servers=127.0.206.84:32825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 13:11:03.927948 20576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:03.928134 20576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:03.928156 20576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:03.929270 20576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:03.929309 20576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:03.929333 20576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260925 13:11:03.930653 20576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:32825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:44353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:03.930905 20576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:03.931190 20576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:03.936030 20581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:03.936046 20584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:03.936012 20582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:03.936223 20576 server_base.cc:1034] running on GCE node
I20260925 13:11:03.936530 20576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:03.936860 20576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:03.938046 20576 hybrid_clock.cc:648] HybridClock initialized: now 1790341863938028 us; error 44 us; skew 500 ppm
I20260925 13:11:03.939400 20576 webserver.cc:460] Webserver started at http://127.0.206.65:46677/ using document root <none> and password file <none>
I20260925 13:11:03.939657 20576 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:03.939710 20576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:03.939831 20576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:03.941082 20576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-0/data/instance:
uuid: "060f38c927bd4f1eb1b28418ea790e91"
format_stamp: "Formatted at 2026-09-25 13:11:03 on dist-test-slave-chkf"
server_key: "ffd1e138505d84216abeba9349106dce"
server_key_iv: "103bb7fde4389496180cb3d77e8ea69f"
server_key_version: "encryptionkey@0"
I20260925 13:11:03.941428 20576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-0/wal/instance:
uuid: "060f38c927bd4f1eb1b28418ea790e91"
format_stamp: "Formatted at 2026-09-25 13:11:03 on dist-test-slave-chkf"
server_key: "ffd1e138505d84216abeba9349106dce"
server_key_iv: "103bb7fde4389496180cb3d77e8ea69f"
server_key_version: "encryptionkey@0"
I20260925 13:11:03.942919 20576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260925 13:11:03.943737 20590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:03.943902 20576 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:11:03.943980 20576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-0/wal
uuid: "060f38c927bd4f1eb1b28418ea790e91"
format_stamp: "Formatted at 2026-09-25 13:11:03 on dist-test-slave-chkf"
server_key: "ffd1e138505d84216abeba9349106dce"
server_key_iv: "103bb7fde4389496180cb3d77e8ea69f"
server_key_version: "encryptionkey@0"
I20260925 13:11:03.944062 20576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:03.958374 20576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:03.958621 20576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:03.958792 20576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:03.958992 20576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:03.959312 20576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:03.959344 20576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:03.959383 20576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:03.959409 20576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:03.964818 20576 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:35805
I20260925 13:11:03.964888 20703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:35805 every 8 connection(s)
I20260925 13:11:03.965183 20576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-0/data/info.pb
I20260925 13:11:03.966913 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 20576
I20260925 13:11:03.967026 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-0/wal/instance
I20260925 13:11:03.967267 825 external_mini_cluster.cc:1428] Setting key d5fbcb127a77ae0b409490b9633a47e4
I20260925 13:11:03.968214 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:44353
--builtin_ntp_servers=127.0.206.84:32825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 13:11:03.971204 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.965843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:43291 (local address 127.0.206.126:44353)
0925 13:11:03.965903 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:03.965906 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:03.966148 (+ 242us) server_negotiation.cc:407] Connection header received
0925 13:11:03.966558 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:03.966559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:03.966569 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:03.966619 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:03.967526 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.968195 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.969671 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.969985 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.970060 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.970884 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:03.970886 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:03.970886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:03.970891 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:03.970907 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:03.970911 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:03.970969 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:03.971046 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 13:11:03.971076 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 13:11:03.971278 20706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:03.965842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44353 (local address 127.0.206.65:43291)
0925 13:11:03.966027 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:03.966057 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:03.966486 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:03.966669 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:03.966672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:03.966680 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:03.967406 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:03.967413 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.968289 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.968290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:03.969577 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:03.969580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:03.970068 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:03.970069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:03.970174 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:03.970737 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:11:03.970738 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:03.970781 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:11:03.970803 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:03.970976 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:03.970980 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:03.970983 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:11:03.971033 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 13:11:03.971113 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
I20260925 13:11:03.971762 20704 heartbeater.cc:347] Connected to a master server at 127.0.206.126:44353
I20260925 13:11:03.971874 20704 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:03.972039 20704 heartbeater.cc:511] Master 127.0.206.126:44353 requested a full tablet report, sending...
I20260925 13:11:03.972571 20516 ts_manager.cc:194] Registered new tserver with Master: 060f38c927bd4f1eb1b28418ea790e91 (127.0.206.65:35805)
I20260925 13:11:03.973524 20516 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.65:43291
W20260925 13:11:04.047384 20707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:04.047664 20707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:04.047689 20707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:04.048775 20707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:04.048807 20707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:04.048831 20707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260925 13:11:04.050064 20707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:32825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:44353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:04.050230 20707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:04.050441 20707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:04.055303 20713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:04.055315 20715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:04.055346 20712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:04.055555 20707 server_base.cc:1034] running on GCE node
I20260925 13:11:04.055723 20707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:04.055935 20707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:04.057070 20707 hybrid_clock.cc:648] HybridClock initialized: now 1790341864057058 us; error 27 us; skew 500 ppm
I20260925 13:11:04.058257 20707 webserver.cc:460] Webserver started at http://127.0.206.66:34679/ using document root <none> and password file <none>
I20260925 13:11:04.058477 20707 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:04.058519 20707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:04.058657 20707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:04.059680 20707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-1/data/instance:
uuid: "13b7dd755b4f480386aadfae19b12979"
format_stamp: "Formatted at 2026-09-25 13:11:04 on dist-test-slave-chkf"
server_key: "182c697b48f5ad09e2eceee6976e7a4b"
server_key_iv: "2f0f34a6992578e9c1452a430d4bf29f"
server_key_version: "encryptionkey@0"
I20260925 13:11:04.060012 20707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-1/wal/instance:
uuid: "13b7dd755b4f480386aadfae19b12979"
format_stamp: "Formatted at 2026-09-25 13:11:04 on dist-test-slave-chkf"
server_key: "182c697b48f5ad09e2eceee6976e7a4b"
server_key_iv: "2f0f34a6992578e9c1452a430d4bf29f"
server_key_version: "encryptionkey@0"
I20260925 13:11:04.061246 20707 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:11:04.061983 20721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:04.062161 20707 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:11:04.062258 20707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-1/wal
uuid: "13b7dd755b4f480386aadfae19b12979"
format_stamp: "Formatted at 2026-09-25 13:11:04 on dist-test-slave-chkf"
server_key: "182c697b48f5ad09e2eceee6976e7a4b"
server_key_iv: "2f0f34a6992578e9c1452a430d4bf29f"
server_key_version: "encryptionkey@0"
I20260925 13:11:04.062328 20707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:04.076586 20707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:04.076834 20707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:04.076973 20707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:04.077186 20707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:04.077478 20707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:04.077529 20707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:04.077560 20707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:04.077587 20707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:04.083081 20707 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:34075
I20260925 13:11:04.083130 20834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:34075 every 8 connection(s)
I20260925 13:11:04.083426 20707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-1/data/info.pb
I20260925 13:11:04.088146 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:04.084084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:48185 (local address 127.0.206.126:44353)
0925 13:11:04.084134 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:04.084136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:04.084360 (+ 224us) server_negotiation.cc:407] Connection header received
0925 13:11:04.084814 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:04.084815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:04.084824 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:04.084879 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:04.085487 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.085959 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.086935 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.087106 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.087164 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:04.087932 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:04.087932 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:04.087933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:04.087936 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:04.087945 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:04.087948 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:04.087985 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:04.088065 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 13:11:04.088085 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:11:04.088368 20837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:04.084060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44353 (local address 127.0.206.66:48185)
0925 13:11:04.084248 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:04.084287 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:04.084729 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:04.084951 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:04.084954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:04.084962 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:04.085420 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:04.085424 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.086034 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.086035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:04.086869 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:04.086871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.087186 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.087187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:04.087286 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:04.087816 (+ 530us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:11:04.087816 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:04.087843 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:11:04.087863 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:04.088000 (+ 137us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:04.088004 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:04.088006 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:11:04.088057 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 13:11:04.088131 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
I20260925 13:11:04.088876 20835 heartbeater.cc:347] Connected to a master server at 127.0.206.126:44353
I20260925 13:11:04.088970 20835 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:04.089154 20835 heartbeater.cc:511] Master 127.0.206.126:44353 requested a full tablet report, sending...
I20260925 13:11:04.089529 20516 ts_manager.cc:194] Registered new tserver with Master: 13b7dd755b4f480386aadfae19b12979 (127.0.206.66:34075)
I20260925 13:11:04.089900 20516 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.66:48185
I20260925 13:11:04.093449 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 20707
I20260925 13:11:04.093536 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-1/wal/instance
I20260925 13:11:04.093791 825 external_mini_cluster.cc:1428] Setting key 3206435162df8723c8c6c4ccbd445061
I20260925 13:11:04.094682 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:44353
--builtin_ntp_servers=127.0.206.84:32825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 13:11:04.174427 20838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:04.174623 20838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:04.174643 20838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:04.175853 20838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:04.175891 20838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:04.175915 20838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260925 13:11:04.177265 20838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:32825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:44353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:04.177431 20838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:04.177662 20838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:04.182331 20844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:04.182412 20838 server_base.cc:1034] running on GCE node
W20260925 13:11:04.182309 20843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:04.182315 20846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:04.182744 20838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:04.182987 20838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:04.184144 20838 hybrid_clock.cc:648] HybridClock initialized: now 1790341864184130 us; error 26 us; skew 500 ppm
I20260925 13:11:04.185418 20838 webserver.cc:460] Webserver started at http://127.0.206.67:43397/ using document root <none> and password file <none>
I20260925 13:11:04.185675 20838 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:04.185745 20838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:04.185883 20838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:04.186936 20838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-2/data/instance:
uuid: "fad668004e084fbab756dc115b80de23"
format_stamp: "Formatted at 2026-09-25 13:11:04 on dist-test-slave-chkf"
server_key: "b6f0aa93a85d86fddcd3f9fb35b3dabf"
server_key_iv: "47d1e785668551cbad824a1a4a57742f"
server_key_version: "encryptionkey@0"
I20260925 13:11:04.187271 20838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-2/wal/instance:
uuid: "fad668004e084fbab756dc115b80de23"
format_stamp: "Formatted at 2026-09-25 13:11:04 on dist-test-slave-chkf"
server_key: "b6f0aa93a85d86fddcd3f9fb35b3dabf"
server_key_iv: "47d1e785668551cbad824a1a4a57742f"
server_key_version: "encryptionkey@0"
I20260925 13:11:04.188478 20838 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:11:04.189160 20852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:04.189327 20838 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:11:04.189420 20838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-2/wal
uuid: "fad668004e084fbab756dc115b80de23"
format_stamp: "Formatted at 2026-09-25 13:11:04 on dist-test-slave-chkf"
server_key: "b6f0aa93a85d86fddcd3f9fb35b3dabf"
server_key_iv: "47d1e785668551cbad824a1a4a57742f"
server_key_version: "encryptionkey@0"
I20260925 13:11:04.189497 20838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:04.202037 20838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:04.202332 20838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:04.202474 20838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:04.202683 20838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 13:11:04.203032 20838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 13:11:04.203087 20838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:04.203125 20838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 13:11:04.203150 20838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:04.208698 20838 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:36391
I20260925 13:11:04.208734 20965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:36391 every 8 connection(s)
I20260925 13:11:04.209035 20838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-2/data/info.pb
I20260925 13:11:04.209334 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 20838
I20260925 13:11:04.209455 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-2/wal/instance
I20260925 13:11:04.209668 825 external_mini_cluster.cc:1428] Setting key 9cda80b98277acd7f6f9d3d11f99f095
I20260925 13:11:04.213918 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:04.209801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:53721 (local address 127.0.206.126:44353)
0925 13:11:04.209846 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:04.209848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:04.210135 (+ 287us) server_negotiation.cc:407] Connection header received
0925 13:11:04.210801 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:04.210804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:04.210819 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:04.210858 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:04.211436 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.211892 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.212794 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.212967 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.213015 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:04.213700 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:04.213701 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:04.213702 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:04.213706 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:04.213717 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:04.213720 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:04.213762 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:04.213842 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 13:11:04.213864 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 13:11:04.214111 20968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:04.209787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44353 (local address 127.0.206.67:53721)
0925 13:11:04.210008 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:04.210047 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:04.210666 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:04.210914 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:04.210917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:04.210924 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:04.211370 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:04.211374 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.211965 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.211966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:04.212723 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:04.212724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.213038 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.213039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:04.213146 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:04.213578 (+ 432us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 13:11:04.213579 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:04.213603 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 13:11:04.213624 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:04.213773 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:04.213777 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:04.213779 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 13:11:04.213827 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 13:11:04.213899 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":86,"threads_started":1}
I20260925 13:11:04.214548 20966 heartbeater.cc:347] Connected to a master server at 127.0.206.126:44353
I20260925 13:11:04.214641 20966 heartbeater.cc:464] Registering TS with master...
I20260925 13:11:04.214831 20966 heartbeater.cc:511] Master 127.0.206.126:44353 requested a full tablet report, sending...
I20260925 13:11:04.215132 20516 ts_manager.cc:194] Registered new tserver with Master: fad668004e084fbab756dc115b80de23 (127.0.206.67:36391)
I20260925 13:11:04.215476 20516 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.67:53721
I20260925 13:11:04.221434 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 13:11:04.225455 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:04.222679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47524 (local address 127.0.206.126:44353)
0925 13:11:04.222763 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:04.222765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:04.222865 (+ 100us) server_negotiation.cc:407] Connection header received
0925 13:11:04.222935 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:04.222936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:04.222946 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:04.223015 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:04.223203 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.223651 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.224381 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.224569 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.224658 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:04.225236 (+ 578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:04.225237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:04.225237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:04.225242 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:04.225253 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:04.225257 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:04.225302 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:04.225356 (+ 54us) server_negotiation.cc:299] Negotiation successful
0925 13:11:04.225378 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 13:11:04.226874 20516 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47524:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 13:11:04.228327 20516 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 13:11:04.234230 20975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:04.230436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37828 (local address 127.0.206.67:36391)
0925 13:11:04.230846 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:04.230849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:04.230865 (+ 16us) server_negotiation.cc:407] Connection header received
0925 13:11:04.230891 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:04.230892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:04.230903 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:04.230987 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:04.231199 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.232024 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.233177 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.233867 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.233954 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:04.234035 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:04.234119 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 13:11:04.234154 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":67,"threads_started":1}
I20260925 13:11:04.234223 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:04.230402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:36391 (local address 127.0.0.1:37828)
0925 13:11:04.230580 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:04.230622 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:04.230683 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:04.231013 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:04.231017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:04.231019 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:04.231085 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:04.231087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.232109 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.232110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:04.233084 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:04.233087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.233954 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.233955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:04.234025 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:04.234036 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 13:11:04.234106 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 13:11:04.234160 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":61,"threads_started":1}
I20260925 13:11:04.234875 20976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:04.230440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:34075 (local address 127.0.0.1:36182)
0925 13:11:04.231085 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:04.231095 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:04.231144 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:04.231395 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:04.231396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:04.231398 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:04.231436 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:04.231438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.232698 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.232699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:04.233743 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:04.233746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.234603 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.234604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:04.234675 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:04.234680 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 13:11:04.234775 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 13:11:04.234813 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":78,"threads_started":1}
I20260925 13:11:04.234958 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:04.230562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36182 (local address 127.0.206.66:34075)
0925 13:11:04.230778 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:04.230782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:04.231146 (+ 364us) server_negotiation.cc:407] Connection header received
0925 13:11:04.231236 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:04.231240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:04.231261 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:04.231334 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:04.231512 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.232609 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.233831 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.234517 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.234602 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:04.234687 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:04.234817 (+ 130us) server_negotiation.cc:299] Negotiation successful
0925 13:11:04.234858 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":84,"threads_started":1}
I20260925 13:11:04.235143 20979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:04.230696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57726 (local address 127.0.206.65:35805)
0925 13:11:04.230873 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:04.230877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:04.231348 (+ 471us) server_negotiation.cc:407] Connection header received
0925 13:11:04.231393 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:04.231395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:04.231410 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:04.231490 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:04.231626 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.232707 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.233710 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.234590 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.234649 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:04.234744 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:04.235027 (+ 283us) server_negotiation.cc:299] Negotiation successful
0925 13:11:04.235079 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260925 13:11:04.235085 20900 tablet_service.cc:1467] Processing CreateTablet for tablet 46dff5516abc4d0db77edd8cd52311db (DEFAULT_TABLE table=test-table [id=1773bbc285c14cdb8fdcf4ef5f396e9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:04.235360 20978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:04.230599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:35805 (local address 127.0.0.1:57726)
0925 13:11:04.231234 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:04.231242 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:04.231283 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:04.231500 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:04.231501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:04.231504 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:04.231544 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:04.231545 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.232769 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.232770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:04.233624 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:04.233626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.234787 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.234789 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:04.234849 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:04.234852 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:11:04.234909 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 13:11:04.235278 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":73,"threads_started":1}
I20260925 13:11:04.235409 20900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46dff5516abc4d0db77edd8cd52311db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:04.235600 20769 tablet_service.cc:1467] Processing CreateTablet for tablet 46dff5516abc4d0db77edd8cd52311db (DEFAULT_TABLE table=test-table [id=1773bbc285c14cdb8fdcf4ef5f396e9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:04.235914 20769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46dff5516abc4d0db77edd8cd52311db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:04.236021 20638 tablet_service.cc:1467] Processing CreateTablet for tablet 46dff5516abc4d0db77edd8cd52311db (DEFAULT_TABLE table=test-table [id=1773bbc285c14cdb8fdcf4ef5f396e9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 13:11:04.236255 20638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46dff5516abc4d0db77edd8cd52311db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:04.238855 20981 tablet_bootstrap.cc:492] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23: Bootstrap starting.
I20260925 13:11:04.239099 20982 tablet_bootstrap.cc:492] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979: Bootstrap starting.
I20260925 13:11:04.239240 20980 tablet_bootstrap.cc:492] T 46dff5516abc4d0db77edd8cd52311db P 060f38c927bd4f1eb1b28418ea790e91: Bootstrap starting.
I20260925 13:11:04.239532 20981 tablet_bootstrap.cc:654] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:04.239792 20982 tablet_bootstrap.cc:654] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:04.239813 20980 tablet_bootstrap.cc:654] T 46dff5516abc4d0db77edd8cd52311db P 060f38c927bd4f1eb1b28418ea790e91: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:04.239823 20981 log.cc:824] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:04.240125 20980 log.cc:824] T 46dff5516abc4d0db77edd8cd52311db P 060f38c927bd4f1eb1b28418ea790e91: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:04.240124 20982 log.cc:824] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:04.240581 20981 tablet_bootstrap.cc:492] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23: No bootstrap required, opened a new log
I20260925 13:11:04.240695 20981 ts_tablet_manager.cc:1397] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 13:11:04.240801 20980 tablet_bootstrap.cc:492] T 46dff5516abc4d0db77edd8cd52311db P 060f38c927bd4f1eb1b28418ea790e91: No bootstrap required, opened a new log
I20260925 13:11:04.240852 20980 ts_tablet_manager.cc:1397] T 46dff5516abc4d0db77edd8cd52311db P 060f38c927bd4f1eb1b28418ea790e91: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:11:04.240880 20982 tablet_bootstrap.cc:492] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979: No bootstrap required, opened a new log
I20260925 13:11:04.240942 20982 ts_tablet_manager.cc:1397] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 13:11:04.242673 20980 raft_consensus.cc:348] T 46dff5516abc4d0db77edd8cd52311db P 060f38c927bd4f1eb1b28418ea790e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fad668004e084fbab756dc115b80de23" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36391 } } peers { permanent_uuid: "060f38c927bd4f1eb1b28418ea790e91" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35805 } } peers { permanent_uuid: "13b7dd755b4f480386aadfae19b12979" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34075 } }
I20260925 13:11:04.242851 20980 raft_consensus.cc:374] T 46dff5516abc4d0db77edd8cd52311db P 060f38c927bd4f1eb1b28418ea790e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:04.242883 20980 raft_consensus.cc:729] T 46dff5516abc4d0db77edd8cd52311db P 060f38c927bd4f1eb1b28418ea790e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 060f38c927bd4f1eb1b28418ea790e91, State: Initialized, Role: FOLLOWER
I20260925 13:11:04.242987 20980 consensus_queue.cc:260] T 46dff5516abc4d0db77edd8cd52311db P 060f38c927bd4f1eb1b28418ea790e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fad668004e084fbab756dc115b80de23" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36391 } } peers { permanent_uuid: "060f38c927bd4f1eb1b28418ea790e91" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35805 } } peers { permanent_uuid: "13b7dd755b4f480386aadfae19b12979" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34075 } }
I20260925 13:11:04.243242 20980 ts_tablet_manager.cc:1428] T 46dff5516abc4d0db77edd8cd52311db P 060f38c927bd4f1eb1b28418ea790e91: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 13:11:04.243260 20704 heartbeater.cc:503] Master 127.0.206.126:44353 was elected leader, sending a full tablet report...
I20260925 13:11:04.243431 20982 raft_consensus.cc:348] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fad668004e084fbab756dc115b80de23" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36391 } } peers { permanent_uuid: "060f38c927bd4f1eb1b28418ea790e91" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35805 } } peers { permanent_uuid: "13b7dd755b4f480386aadfae19b12979" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34075 } }
I20260925 13:11:04.243589 20982 raft_consensus.cc:374] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:04.243646 20982 raft_consensus.cc:729] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13b7dd755b4f480386aadfae19b12979, State: Initialized, Role: FOLLOWER
I20260925 13:11:04.243769 20982 consensus_queue.cc:260] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fad668004e084fbab756dc115b80de23" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36391 } } peers { permanent_uuid: "060f38c927bd4f1eb1b28418ea790e91" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35805 } } peers { permanent_uuid: "13b7dd755b4f480386aadfae19b12979" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34075 } }
I20260925 13:11:04.243813 20981 raft_consensus.cc:348] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fad668004e084fbab756dc115b80de23" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36391 } } peers { permanent_uuid: "060f38c927bd4f1eb1b28418ea790e91" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35805 } } peers { permanent_uuid: "13b7dd755b4f480386aadfae19b12979" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34075 } }
I20260925 13:11:04.243954 20981 raft_consensus.cc:374] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:04.243988 20981 raft_consensus.cc:729] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fad668004e084fbab756dc115b80de23, State: Initialized, Role: FOLLOWER
I20260925 13:11:04.244011 20982 ts_tablet_manager.cc:1428] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 13:11:04.244050 20835 heartbeater.cc:503] Master 127.0.206.126:44353 was elected leader, sending a full tablet report...
I20260925 13:11:04.244091 20981 consensus_queue.cc:260] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fad668004e084fbab756dc115b80de23" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36391 } } peers { permanent_uuid: "060f38c927bd4f1eb1b28418ea790e91" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35805 } } peers { permanent_uuid: "13b7dd755b4f480386aadfae19b12979" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34075 } }
I20260925 13:11:04.244333 20981 ts_tablet_manager.cc:1428] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260925 13:11:04.244450 20966 heartbeater.cc:503] Master 127.0.206.126:44353 was elected leader, sending a full tablet report...
W20260925 13:11:04.334015 20836 tablet.cc:2369] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:11:04.459585 20967 tablet.cc:2369] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 13:11:04.465795 20705 tablet.cc:2369] T 46dff5516abc4d0db77edd8cd52311db P 060f38c927bd4f1eb1b28418ea790e91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 13:11:04.555727 20987 raft_consensus.cc:482] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 13:11:04.555847 20987 raft_consensus.cc:504] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fad668004e084fbab756dc115b80de23" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36391 } } peers { permanent_uuid: "060f38c927bd4f1eb1b28418ea790e91" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35805 } } peers { permanent_uuid: "13b7dd755b4f480386aadfae19b12979" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34075 } }
I20260925 13:11:04.556134 20987 leader_election.cc:288] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fad668004e084fbab756dc115b80de23 (127.0.206.67:36391), 060f38c927bd4f1eb1b28418ea790e91 (127.0.206.65:35805)
I20260925 13:11:04.559036 20975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:04.556469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:60973 (local address 127.0.206.67:36391)
0925 13:11:04.556539 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:04.556541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:04.556552 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:11:04.556588 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:04.556588 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:04.556598 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:04.556656 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:04.556824 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.557307 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.558251 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.558625 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.558710 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:04.558735 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:04.558894 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 13:11:04.558930 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 13:11:04.559052 20837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:04.556355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:36391 (local address 127.0.206.66:60973)
0925 13:11:04.556415 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:04.556428 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:04.556515 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:04.556674 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:04.556674 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:04.556676 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:04.556746 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:04.556747 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.557391 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.557392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:04.558153 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:04.558155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.558726 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.558727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:04.558797 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:04.558800 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:11:04.558887 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 13:11:04.558963 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":1}
I20260925 13:11:04.559303 20979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:04.556551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:57381 (local address 127.0.206.65:35805)
0925 13:11:04.556592 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:04.556594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:04.556707 (+ 113us) server_negotiation.cc:407] Connection header received
0925 13:11:04.556781 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:04.556782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:04.556799 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:04.556859 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 13:11:04.557020 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.557478 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.558409 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.558837 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.558921 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:04.558929 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:04.559184 (+ 255us) server_negotiation.cc:299] Negotiation successful
0925 13:11:04.559233 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 13:11:04.559383 20920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46dff5516abc4d0db77edd8cd52311db" candidate_uuid: "13b7dd755b4f480386aadfae19b12979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fad668004e084fbab756dc115b80de23" is_pre_election: true
I20260925 13:11:04.559532 20989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:04.556465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:35805 (local address 127.0.206.66:57381)
0925 13:11:04.556648 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:04.556655 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:04.556716 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:04.556861 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:04.556862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:04.556863 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 13:11:04.556949 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:04.556950 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.557561 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.557562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:04.558310 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:04.558312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.559008 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.559009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:04.559079 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:04.559082 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 13:11:04.559414 (+ 332us) client_negotiation.cc:240] Negotiation successful
0925 13:11:04.559459 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":10,"thread_start_us":101,"threads_started":1}
I20260925 13:11:04.559540 20920 raft_consensus.cc:2454] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13b7dd755b4f480386aadfae19b12979 in term 0.
I20260925 13:11:04.559777 20725 leader_election.cc:302] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13b7dd755b4f480386aadfae19b12979, fad668004e084fbab756dc115b80de23; no voters:
I20260925 13:11:04.559842 20658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46dff5516abc4d0db77edd8cd52311db" candidate_uuid: "13b7dd755b4f480386aadfae19b12979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "060f38c927bd4f1eb1b28418ea790e91" is_pre_election: true
I20260925 13:11:04.559926 20987 raft_consensus.cc:2781] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 13:11:04.559988 20987 raft_consensus.cc:482] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 13:11:04.560004 20658 raft_consensus.cc:2454] T 46dff5516abc4d0db77edd8cd52311db P 060f38c927bd4f1eb1b28418ea790e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13b7dd755b4f480386aadfae19b12979 in term 0.
I20260925 13:11:04.560019 20987 raft_consensus.cc:3037] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:04.560768 20987 raft_consensus.cc:504] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fad668004e084fbab756dc115b80de23" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36391 } } peers { permanent_uuid: "060f38c927bd4f1eb1b28418ea790e91" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35805 } } peers { permanent_uuid: "13b7dd755b4f480386aadfae19b12979" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34075 } }
I20260925 13:11:04.560904 20987 leader_election.cc:288] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [CANDIDATE]: Term 1 election: Requested vote from peers fad668004e084fbab756dc115b80de23 (127.0.206.67:36391), 060f38c927bd4f1eb1b28418ea790e91 (127.0.206.65:35805)
I20260925 13:11:04.561059 20920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46dff5516abc4d0db77edd8cd52311db" candidate_uuid: "13b7dd755b4f480386aadfae19b12979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fad668004e084fbab756dc115b80de23"
I20260925 13:11:04.561074 20658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46dff5516abc4d0db77edd8cd52311db" candidate_uuid: "13b7dd755b4f480386aadfae19b12979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "060f38c927bd4f1eb1b28418ea790e91"
I20260925 13:11:04.561158 20920 raft_consensus.cc:3037] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:04.561157 20658 raft_consensus.cc:3037] T 46dff5516abc4d0db77edd8cd52311db P 060f38c927bd4f1eb1b28418ea790e91 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:04.561847 20658 raft_consensus.cc:2454] T 46dff5516abc4d0db77edd8cd52311db P 060f38c927bd4f1eb1b28418ea790e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13b7dd755b4f480386aadfae19b12979 in term 1.
I20260925 13:11:04.561872 20920 raft_consensus.cc:2454] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13b7dd755b4f480386aadfae19b12979 in term 1.
I20260925 13:11:04.562037 20725 leader_election.cc:302] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 060f38c927bd4f1eb1b28418ea790e91, 13b7dd755b4f480386aadfae19b12979; no voters:
I20260925 13:11:04.562144 20987 raft_consensus.cc:2781] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:04.562282 20987 raft_consensus.cc:686] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [term 1 LEADER]: Becoming Leader. State: Replica: 13b7dd755b4f480386aadfae19b12979, State: Running, Role: LEADER
I20260925 13:11:04.562372 20987 consensus_queue.cc:237] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fad668004e084fbab756dc115b80de23" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36391 } } peers { permanent_uuid: "060f38c927bd4f1eb1b28418ea790e91" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35805 } } peers { permanent_uuid: "13b7dd755b4f480386aadfae19b12979" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34075 } }
I20260925 13:11:04.563179 20515 catalog_manager.cc:5495] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13b7dd755b4f480386aadfae19b12979 (127.0.206.66). New cstate: current_term: 1 leader_uuid: "13b7dd755b4f480386aadfae19b12979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fad668004e084fbab756dc115b80de23" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "060f38c927bd4f1eb1b28418ea790e91" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13b7dd755b4f480386aadfae19b12979" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34075 } health_report { overall_health: HEALTHY } } }
I20260925 13:11:04.584568 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:04.582505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36190 (local address 127.0.206.66:34075)
0925 13:11:04.582540 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:04.582542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:04.582549 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:11:04.582621 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:04.582622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:04.582635 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:04.582692 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:04.582842 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.583260 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.583911 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:04.584080 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:04.584133 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:04.584250 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:04.584332 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:04.584386 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:04.584455 (+ 69us) server_negotiation.cc:299] Negotiation successful
0925 13:11:04.584467 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:11:04.586154 20658 raft_consensus.cc:1261] T 46dff5516abc4d0db77edd8cd52311db P 060f38c927bd4f1eb1b28418ea790e91 [term 1 FOLLOWER]: Refusing update from remote peer 13b7dd755b4f480386aadfae19b12979: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:04.586233 20920 raft_consensus.cc:1261] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23 [term 1 FOLLOWER]: Refusing update from remote peer 13b7dd755b4f480386aadfae19b12979: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:04.586548 20987 consensus_queue.cc:1035] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "060f38c927bd4f1eb1b28418ea790e91" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:04.586721 20990 consensus_queue.cc:1035] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fad668004e084fbab756dc115b80de23" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:04.588879 20993 mvcc.cc:205] Tried to move back new op lower bound from 7333240277342261248 to 7333240277247963136. Current Snapshot: MvccSnapshot[applied={T|T < 7333240277342261248}]
I20260925 13:11:04.589141 20994 mvcc.cc:205] Tried to move back new op lower bound from 7333240277342261248 to 7333240277247963136. Current Snapshot: MvccSnapshot[applied={T|T < 7333240277342261248}]
I20260925 13:11:04.589178 20995 mvcc.cc:205] Tried to move back new op lower bound from 7333240277342261248 to 7333240277247963136. Current Snapshot: MvccSnapshot[applied={T|T < 7333240277342261248}]
I20260925 13:11:04.591239 20514 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:11:04.591315 20514 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 13:11:04.592036 20514 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bec179aa1bfc4b4db0f061fb776972d0: Sending DeleteTablet for 3 replicas of tablet 46dff5516abc4d0db77edd8cd52311db
I20260925 13:11:04.592352 20900 tablet_service.cc:1514] Processing DeleteTablet for tablet 46dff5516abc4d0db77edd8cd52311db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:11:04 UTC) from {username='slave'} at 127.0.0.1:37828
I20260925 13:11:04.592391 20638 tablet_service.cc:1514] Processing DeleteTablet for tablet 46dff5516abc4d0db77edd8cd52311db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:11:04 UTC) from {username='slave'} at 127.0.0.1:57726
I20260925 13:11:04.592572 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 20576
I20260925 13:11:04.592653 20769 tablet_service.cc:1514] Processing DeleteTablet for tablet 46dff5516abc4d0db77edd8cd52311db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 13:11:04 UTC) from {username='slave'} at 127.0.0.1:36182
I20260925 13:11:04.592623 21003 tablet_replica.cc:330] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23: stopping tablet replica
I20260925 13:11:04.593195 21003 raft_consensus.cc:2229] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 13:11:04.593282 21003 raft_consensus.cc:2258] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:04.593717 21003 ts_tablet_manager.cc:1905] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:11:04.595268 21005 tablet_replica.cc:330] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979: stopping tablet replica
I20260925 13:11:04.595278 21003 ts_tablet_manager.cc:1918] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:11:04.595345 21003 log.cc:1196] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-2/wal/wals/46dff5516abc4d0db77edd8cd52311db
I20260925 13:11:04.595378 21005 raft_consensus.cc:2229] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [term 1 LEADER]: Raft consensus shutting down.
I20260925 13:11:04.595469 21005 raft_consensus.cc:2258] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 13:11:04.595607 21003 ts_tablet_manager.cc:1939] T 46dff5516abc4d0db77edd8cd52311db P fad668004e084fbab756dc115b80de23: Deleting consensus metadata
I20260925 13:11:04.595872 21005 ts_tablet_manager.cc:1905] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 13:11:04.596029 20503 catalog_manager.cc:4841] TS fad668004e084fbab756dc115b80de23 (127.0.206.67:36391): tablet 46dff5516abc4d0db77edd8cd52311db (table test-table [id=1773bbc285c14cdb8fdcf4ef5f396e9c]) successfully deleted
I20260925 13:11:04.597131 21005 ts_tablet_manager.cc:1918] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 13:11:04.597201 21005 log.cc:1196] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979: Deleting WAL directory at /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790341627249256-825-0/minicluster-data/ts-1/wal/wals/46dff5516abc4d0db77edd8cd52311db
I20260925 13:11:04.597434 21005 ts_tablet_manager.cc:1939] T 46dff5516abc4d0db77edd8cd52311db P 13b7dd755b4f480386aadfae19b12979: Deleting consensus metadata
I20260925 13:11:04.597790 20500 catalog_manager.cc:4841] TS 13b7dd755b4f480386aadfae19b12979 (127.0.206.66:34075): tablet 46dff5516abc4d0db77edd8cd52311db (table test-table [id=1773bbc285c14cdb8fdcf4ef5f396e9c]) successfully deleted
W20260925 13:11:04.605400 20503 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:35805 (error 108)
I20260925 13:11:04.605664 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 20707
I20260925 13:11:04.606380 20978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:04.605822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:35805 (local address 127.0.0.1:57732)
0925 13:11:04.606302 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:04.606345 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456}
W20260925 13:11:04.606796 20503 catalog_manager.cc:4584] TS 060f38c927bd4f1eb1b28418ea790e91 (127.0.206.65:35805): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 46dff5516abc4d0db77edd8cd52311db: Network error: Client connection negotiation failed: client connection to 127.0.206.65:35805: connect: Connection refused (error 111)
I20260925 13:11:04.610571 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 20838
I20260925 13:11:04.614948 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 20485
2026-09-25T13:11:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (960 ms)
[----------] 6 tests from EncryptionPolicyTest (8951 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:04 dist-test-slave-chkf krb5kdc[21009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:04 dist-test-slave-chkf krb5kdc[21009](info): set up 2 sockets
Sep 25 13:11:04 dist-test-slave-chkf krb5kdc[21009](info): commencing operation
krb5kdc: starting...
W20260925 13:11:05.642620 825 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:05 dist-test-slave-chkf krb5kdc[21009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 13:11:05.754024 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38811
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:05.832520 21022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:05.832770 21022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:05.832828 21022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:05.834038 21022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:05.834080 21022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:05.834095 21022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:05.834113 21022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:05.834126 21022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:05.835623 21022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38811
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:05.835858 21022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:05.836086 21022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:05.840925 21027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:05.840901 21028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:05.840946 21022 server_base.cc:1034] running on GCE node
W20260925 13:11:05.840917 21030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:05.841395 21022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:05.841684 21022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:05.842844 21022 hybrid_clock.cc:648] HybridClock initialized: now 1790341865842827 us; error 27 us; skew 500 ppm
Sep 25 13:11:05 dist-test-slave-chkf krb5kdc[21009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341865, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:05.848832 21022 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 13:11:05.849258 21022 webserver.cc:460] Webserver started at http://127.0.0.1:36457/ using document root <none> and password file <none>
I20260925 13:11:05.849470 21022 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:05.849537 21022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:05.849687 21022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:05.850869 21022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "960dd5a5a91d43c5bad6409078ccf29d"
format_stamp: "Formatted at 2026-09-25 13:11:05 on dist-test-slave-chkf"
server_key: "5abddefc5471a7beb801b14ee1a4e878"
server_key_iv: "93f1d3f54bdef156e75f18731a4d3056"
server_key_version: "encryptionkey@0"
I20260925 13:11:05.851224 21022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "960dd5a5a91d43c5bad6409078ccf29d"
format_stamp: "Formatted at 2026-09-25 13:11:05 on dist-test-slave-chkf"
server_key: "5abddefc5471a7beb801b14ee1a4e878"
server_key_iv: "93f1d3f54bdef156e75f18731a4d3056"
server_key_version: "encryptionkey@0"
I20260925 13:11:05.852483 21022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:11:05.853168 21037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:05.853328 21022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:11:05.853427 21022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "960dd5a5a91d43c5bad6409078ccf29d"
format_stamp: "Formatted at 2026-09-25 13:11:05 on dist-test-slave-chkf"
server_key: "5abddefc5471a7beb801b14ee1a4e878"
server_key_iv: "93f1d3f54bdef156e75f18731a4d3056"
server_key_version: "encryptionkey@0"
I20260925 13:11:05.853506 21022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:05.861129 21022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:05.862972 21022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:05.863126 21022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:05.866855 21022 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38811
I20260925 13:11:05.866909 21089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38811 every 8 connection(s)
I20260925 13:11:05.867246 21022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:05.868093 21090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:05.868175 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 21022
I20260925 13:11:05.868299 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:05.868512 825 external_mini_cluster.cc:1428] Setting key 7097f4d67e5b8d94922b9b64cb8ec252
I20260925 13:11:05.871483 21090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d: Bootstrap starting.
I20260925 13:11:05.872150 21090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:05.872439 21090 log.cc:824] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:05.873133 21090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d: No bootstrap required, opened a new log
Sep 25 13:11:05 dist-test-slave-chkf krb5kdc[21009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 13:11:05.875821 21090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "960dd5a5a91d43c5bad6409078ccf29d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38811 } }
I20260925 13:11:05.875988 21090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:05.876047 21090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 960dd5a5a91d43c5bad6409078ccf29d, State: Initialized, Role: FOLLOWER
I20260925 13:11:05.876175 21090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "960dd5a5a91d43c5bad6409078ccf29d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38811 } }
I20260925 13:11:05.876255 21090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:05.876292 21090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:05.876332 21090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:05.877017 21090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "960dd5a5a91d43c5bad6409078ccf29d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38811 } }
I20260925 13:11:05.877138 21090 leader_election.cc:302] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 960dd5a5a91d43c5bad6409078ccf29d; no voters:
I20260925 13:11:05.877339 21090 leader_election.cc:288] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:05.877369 21095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:05.877470 21095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d [term 1 LEADER]: Becoming Leader. State: Replica: 960dd5a5a91d43c5bad6409078ccf29d, State: Running, Role: LEADER
I20260925 13:11:05.877547 21095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "960dd5a5a91d43c5bad6409078ccf29d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38811 } }
I20260925 13:11:05.877597 21090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:05.877831 21097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "960dd5a5a91d43c5bad6409078ccf29d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "960dd5a5a91d43c5bad6409078ccf29d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38811 } } }
I20260925 13:11:05.877848 21096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 960dd5a5a91d43c5bad6409078ccf29d. Latest consensus state: current_term: 1 leader_uuid: "960dd5a5a91d43c5bad6409078ccf29d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "960dd5a5a91d43c5bad6409078ccf29d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38811 } } }
I20260925 13:11:05.877928 21097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:05.877948 21096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:05.878137 21100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:05.878619 21100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:05.880563 21093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:05.869011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46149 (local address 127.0.0.1:38811)
0925 13:11:05.869193 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:05.869202 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:05.869234 (+ 32us) server_negotiation.cc:407] Connection header received
0925 13:11:05.869727 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:05.869736 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:05.869798 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:05.869946 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:05.870517 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:05.871966 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:05.872776 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:05.873282 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:05.873393 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:05.876407 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:05.876411 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:05.876412 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:05.876423 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:05.878499 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:05.879133 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:05.879136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:05.879136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:05.879239 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:05.879540 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:05.879541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:05.879541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:05.879839 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:05.879974 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:05.880275 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 13:11:05.880382 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260925 13:11:05.880697 21100 catalog_manager.cc:1347] Generated new cluster ID: 8985f7fd09bd4154bb270961f491e524
I20260925 13:11:05.880733 21100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:05.883689 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:05.891844 21093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:05.885012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38397 (local address 127.0.0.1:38811)
0925 13:11:05.885042 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:05.885043 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:05.885155 (+ 112us) server_negotiation.cc:407] Connection header received
0925 13:11:05.885237 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:05.885238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:05.885246 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:05.885272 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:05.885864 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:05.886565 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:05.887561 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:05.887861 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:05.887893 (+ 32us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:05.889248 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:05.889250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:05.889250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:05.889256 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:05.890695 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:05.891097 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:05.891097 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:05.891098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:05.891141 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:05.891331 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:05.891331 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:05.891332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:05.891429 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:05.891504 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:05.891705 (+ 201us) server_negotiation.cc:299] Negotiation successful
0925 13:11:05.891761 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 13:11:05.904546 21100 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:05.905311 21100 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:05.909780 21100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 960dd5a5a91d43c5bad6409078ccf29d: Generated new TSK 0
I20260925 13:11:05.909982 21100 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:05.912703 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 21022
W20260925 13:11:05.917414 21007 connection.cc:535] client connection to 127.0.0.1:38811 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38811): Connection reset by peer (error 104)
2026-09-25T13:11:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:05 dist-test-slave-chkf krb5kdc[21121](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:05 dist-test-slave-chkf krb5kdc[21121](info): set up 2 sockets
Sep 25 13:11:05 dist-test-slave-chkf krb5kdc[21121](info): commencing operation
krb5kdc: starting...
W20260925 13:11:06.938776 825 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:07 dist-test-slave-chkf krb5kdc[21121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 13:11:07.053774 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45479
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:07.133605 21134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:07.133813 21134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:07.133836 21134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:07.135025 21134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:07.135066 21134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:07.135080 21134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:07.135093 21134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:07.135133 21134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:07.136597 21134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45479
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:07.136847 21134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:07.137069 21134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:07.142058 21140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:07.142067 21142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:07.142174 21134 server_base.cc:1034] running on GCE node
W20260925 13:11:07.142067 21139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:07.142529 21134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:07.142866 21134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:07.144028 21134 hybrid_clock.cc:648] HybridClock initialized: now 1790341867144022 us; error 33 us; skew 500 ppm
Sep 25 13:11:07 dist-test-slave-chkf krb5kdc[21121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341867, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:07.146376 21134 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 13:11:07.146927 21134 webserver.cc:460] Webserver started at http://127.0.0.1:43961/ using document root <none> and password file <none>
I20260925 13:11:07.147172 21134 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:07.147241 21134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:07.147388 21134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:07.148444 21134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "5d1183ce17df4cb98c8e4eb33dbc9803"
format_stamp: "Formatted at 2026-09-25 13:11:07 on dist-test-slave-chkf"
server_key: "a4e78ae62d79df540a1ad2090e81eefe"
server_key_iv: "6786f088b97d7be2d0fb7d367ffa9ca8"
server_key_version: "encryptionkey@0"
I20260925 13:11:07.148810 21134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "5d1183ce17df4cb98c8e4eb33dbc9803"
format_stamp: "Formatted at 2026-09-25 13:11:07 on dist-test-slave-chkf"
server_key: "a4e78ae62d79df540a1ad2090e81eefe"
server_key_iv: "6786f088b97d7be2d0fb7d367ffa9ca8"
server_key_version: "encryptionkey@0"
I20260925 13:11:07.150113 21134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:11:07.150874 21149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:07.151048 21134 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:11:07.151150 21134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "5d1183ce17df4cb98c8e4eb33dbc9803"
format_stamp: "Formatted at 2026-09-25 13:11:07 on dist-test-slave-chkf"
server_key: "a4e78ae62d79df540a1ad2090e81eefe"
server_key_iv: "6786f088b97d7be2d0fb7d367ffa9ca8"
server_key_version: "encryptionkey@0"
I20260925 13:11:07.151233 21134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:07.160794 21134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:07.162635 21134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:07.162834 21134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:07.166795 21134 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45479
I20260925 13:11:07.166827 21201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45479 every 8 connection(s)
I20260925 13:11:07.167172 21134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:07.168007 21202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:07.168277 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 21134
I20260925 13:11:07.168408 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:07.168593 825 external_mini_cluster.cc:1428] Setting key 8ecda0cc0753f57e2030f82324abc4d4
I20260925 13:11:07.171164 21202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803: Bootstrap starting.
I20260925 13:11:07.171839 21202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:07.172122 21202 log.cc:824] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:07.172833 21202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803: No bootstrap required, opened a new log
I20260925 13:11:07.174557 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:07.169135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48415 (local address 127.0.0.1:45479)
0925 13:11:07.169324 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:07.169335 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:07.169369 (+ 34us) server_negotiation.cc:407] Connection header received
0925 13:11:07.169868 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:07.169877 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:07.169938 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:07.170095 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:07.170250 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:07.171616 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:07.172418 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:07.172888 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:07.173013 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:07.173745 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:07.173749 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:07.173749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:07.173900 (+ 151us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:07.173928 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:07.173938 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:07.174015 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:07.174143 (+ 128us) server_negotiation.cc:299] Negotiation successful
0925 13:11:07.174195 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260925 13:11:07.174685 21202 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1183ce17df4cb98c8e4eb33dbc9803" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45479 } }
I20260925 13:11:07.174906 21202 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:07.174954 21202 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d1183ce17df4cb98c8e4eb33dbc9803, State: Initialized, Role: FOLLOWER
I20260925 13:11:07.175094 21202 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1183ce17df4cb98c8e4eb33dbc9803" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45479 } }
I20260925 13:11:07.175194 21202 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:07.175227 21202 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:07.175275 21202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:07.175647 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:07.175948 21202 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1183ce17df4cb98c8e4eb33dbc9803" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45479 } }
I20260925 13:11:07.176034 21202 leader_election.cc:302] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d1183ce17df4cb98c8e4eb33dbc9803; no voters:
I20260925 13:11:07.176213 21202 leader_election.cc:288] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:07.176240 21207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:07.176349 21207 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803 [term 1 LEADER]: Becoming Leader. State: Replica: 5d1183ce17df4cb98c8e4eb33dbc9803, State: Running, Role: LEADER
I20260925 13:11:07.176425 21207 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1183ce17df4cb98c8e4eb33dbc9803" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45479 } }
I20260925 13:11:07.176450 21202 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:07.176749 21207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d1183ce17df4cb98c8e4eb33dbc9803. Latest consensus state: current_term: 1 leader_uuid: "5d1183ce17df4cb98c8e4eb33dbc9803" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1183ce17df4cb98c8e4eb33dbc9803" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45479 } } }
I20260925 13:11:07.176759 21209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d1183ce17df4cb98c8e4eb33dbc9803" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d1183ce17df4cb98c8e4eb33dbc9803" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45479 } } }
I20260925 13:11:07.176831 21207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:07.176913 21209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:07.177146 21216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:07.177655 21216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:07.179461 21216 catalog_manager.cc:1347] Generated new cluster ID: f45bd6aa634c40b3801f6cf06ad80e11
I20260925 13:11:07.179503 21216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:07.186276 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:07.178689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48641 (local address 127.0.0.1:45479)
0925 13:11:07.178749 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:07.178751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:07.182917 (+ 4166us) server_negotiation.cc:407] Connection header received
0925 13:11:07.182997 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:07.182998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:07.183008 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:07.183068 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:07.183237 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:07.183986 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:07.184996 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:07.185293 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:07.185349 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:07.186053 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:07.186054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:07.186054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:07.186058 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:07.186070 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:07.186074 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:07.186121 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:07.186194 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 13:11:07.186204 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 13:11:07.188431 21216 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:07.188908 21216 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:07.193851 21216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d1183ce17df4cb98c8e4eb33dbc9803: Generated new TSK 0
I20260925 13:11:07.194028 21216 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:07.200346 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 21134
W20260925 13:11:07.204664 21119 connection.cc:535] client connection to 127.0.0.1:45479 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45479): Connection reset by peer (error 104)
2026-09-25T13:11:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1287 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:07 dist-test-slave-chkf krb5kdc[21233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:07 dist-test-slave-chkf krb5kdc[21233](info): set up 2 sockets
Sep 25 13:11:07 dist-test-slave-chkf krb5kdc[21233](info): commencing operation
krb5kdc: starting...
W20260925 13:11:08.230688 825 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:08 dist-test-slave-chkf krb5kdc[21233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 13:11:08.345032 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40413
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:08.424680 21246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:08.424870 21246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:08.424892 21246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:08.425940 21246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:08.425974 21246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:08.425988 21246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:08.426002 21246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:08.426014 21246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:08.427388 21246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40413
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:08.427587 21246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:08.427778 21246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:08.432610 21252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:08.432631 21251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:08.432600 21254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:08.432835 21246 server_base.cc:1034] running on GCE node
I20260925 13:11:08.433091 21246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:08.433393 21246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:08.434530 21246 hybrid_clock.cc:648] HybridClock initialized: now 1790341868434513 us; error 49 us; skew 500 ppm
Sep 25 13:11:08 dist-test-slave-chkf krb5kdc[21233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341868, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:08.437144 21246 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 13:11:08.437599 21246 webserver.cc:460] Webserver started at http://127.0.0.1:32977/ using document root <none> and password file <none>
I20260925 13:11:08.437839 21246 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:08.437906 21246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:08.438055 21246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:08.439144 21246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "8af59e5146934c86ba5ca8f882f74cb2"
format_stamp: "Formatted at 2026-09-25 13:11:08 on dist-test-slave-chkf"
server_key: "ff5715b8087f8c81696675eb6f62a4de"
server_key_iv: "fe7de4856dceeb1c9a65202c17fe2b78"
server_key_version: "encryptionkey@0"
I20260925 13:11:08.439496 21246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "8af59e5146934c86ba5ca8f882f74cb2"
format_stamp: "Formatted at 2026-09-25 13:11:08 on dist-test-slave-chkf"
server_key: "ff5715b8087f8c81696675eb6f62a4de"
server_key_iv: "fe7de4856dceeb1c9a65202c17fe2b78"
server_key_version: "encryptionkey@0"
I20260925 13:11:08.440788 21246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:11:08.441488 21261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:08.441664 21246 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:11:08.441753 21246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "8af59e5146934c86ba5ca8f882f74cb2"
format_stamp: "Formatted at 2026-09-25 13:11:08 on dist-test-slave-chkf"
server_key: "ff5715b8087f8c81696675eb6f62a4de"
server_key_iv: "fe7de4856dceeb1c9a65202c17fe2b78"
server_key_version: "encryptionkey@0"
I20260925 13:11:08.441834 21246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:08.445104 21246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:08.447337 21246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:08.447487 21246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:08.451507 21246 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40413
I20260925 13:11:08.451689 21313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40413 every 8 connection(s)
I20260925 13:11:08.451905 21246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:08.452832 21314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:08.455063 21314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2: Bootstrap starting.
I20260925 13:11:08.455613 21314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:08.455891 21314 log.cc:824] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:08.456514 21314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2: No bootstrap required, opened a new log
I20260925 13:11:08.458063 21314 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af59e5146934c86ba5ca8f882f74cb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40413 } }
I20260925 13:11:08.458199 21314 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:08.458221 21314 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8af59e5146934c86ba5ca8f882f74cb2, State: Initialized, Role: FOLLOWER
I20260925 13:11:08.458410 21314 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af59e5146934c86ba5ca8f882f74cb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40413 } }
I20260925 13:11:08.458470 21314 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:08.458508 21314 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:08.458556 21314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:08.459131 21314 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af59e5146934c86ba5ca8f882f74cb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40413 } }
I20260925 13:11:08.459215 21314 leader_election.cc:302] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8af59e5146934c86ba5ca8f882f74cb2; no voters:
I20260925 13:11:08.459385 21314 leader_election.cc:288] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:08.459456 21317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:08.459669 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 21246
I20260925 13:11:08.459798 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:08.459803 21317 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2 [term 1 LEADER]: Becoming Leader. State: Replica: 8af59e5146934c86ba5ca8f882f74cb2, State: Running, Role: LEADER
I20260925 13:11:08.459889 21317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af59e5146934c86ba5ca8f882f74cb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40413 } }
I20260925 13:11:08.459983 825 external_mini_cluster.cc:1428] Setting key d57d3f922255a6ab434c5fc145488ef4
I20260925 13:11:08.460279 21314 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:08.460301 21319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8af59e5146934c86ba5ca8f882f74cb2. Latest consensus state: current_term: 1 leader_uuid: "8af59e5146934c86ba5ca8f882f74cb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af59e5146934c86ba5ca8f882f74cb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40413 } } }
I20260925 13:11:08.460412 21319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:08.460300 21318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8af59e5146934c86ba5ca8f882f74cb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af59e5146934c86ba5ca8f882f74cb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40413 } } }
I20260925 13:11:08.460664 21318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:08.461565 21322 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:08.462059 21322 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:08.462436 21323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:08.460596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33821 (local address 127.0.0.1:40413)
0925 13:11:08.460772 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:08.460781 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:08.461045 (+ 264us) server_negotiation.cc:407] Connection header received
0925 13:11:08.461132 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:08.461137 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:08.461159 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:08.461258 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:08.461972 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:08.461975 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:08.461976 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:08.461980 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:08.462042 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:08.462056 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:08.462119 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:08.462196 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 13:11:08.462237 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":59,"threads_started":1}
I20260925 13:11:08.463532 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:08.463774 21322 catalog_manager.cc:1347] Generated new cluster ID: 5873e5e6a27c46c38a06ccf3c2fe16a5
I20260925 13:11:08.463810 21322 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:08.469543 21323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:08.468323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57237 (local address 127.0.0.1:40413)
0925 13:11:08.468359 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:08.468361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:08.468592 (+ 231us) server_negotiation.cc:407] Connection header received
0925 13:11:08.468672 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:08.468673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:08.468679 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:08.468726 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:08.469324 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:08.469326 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:08.469326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:08.469330 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:08.469344 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:08.469348 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:08.469395 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:08.469454 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 13:11:08.469465 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:11:08.488936 21322 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:08.489116 21322 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:08.502563 21322 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8af59e5146934c86ba5ca8f882f74cb2: Generated new TSK 0
I20260925 13:11:08.502803 21322 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:08.507727 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 21246
2026-09-25T13:11:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1307 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:08 dist-test-slave-chkf krb5kdc[21345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:08 dist-test-slave-chkf krb5kdc[21345](info): set up 2 sockets
Sep 25 13:11:08 dist-test-slave-chkf krb5kdc[21345](info): commencing operation
krb5kdc: starting...
W20260925 13:11:09.546845 825 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:09 dist-test-slave-chkf krb5kdc[21345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 13:11:09.659092 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46203
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:09.737843 21358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:09.738073 21358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:09.738107 21358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:09.739270 21358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:09.739333 21358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:09.739358 21358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:09.739387 21358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:09.739416 21358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:09.740855 21358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:09.741117 21358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:09.741338 21358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:09.745973 21363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:09.746007 21364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:09.746004 21366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:09.746238 21358 server_base.cc:1034] running on GCE node
I20260925 13:11:09.746486 21358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:09.746820 21358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:09.747982 21358 hybrid_clock.cc:648] HybridClock initialized: now 1790341869747970 us; error 47 us; skew 500 ppm
Sep 25 13:11:09 dist-test-slave-chkf krb5kdc[21345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341869, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:09.750355 21358 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 13:11:09.750815 21358 webserver.cc:460] Webserver started at http://127.0.0.1:46197/ using document root <none> and password file <none>
I20260925 13:11:09.751044 21358 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:09.751109 21358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:09.751258 21358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:09.752259 21358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "a4e611be2d144356800ba704025cd250"
format_stamp: "Formatted at 2026-09-25 13:11:09 on dist-test-slave-chkf"
server_key: "7f5e58ccfa6a6cdcd3dd1ff307b6c6ef"
server_key_iv: "685c586d20d286961d9696b146d524c6"
server_key_version: "encryptionkey@0"
I20260925 13:11:09.752599 21358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "a4e611be2d144356800ba704025cd250"
format_stamp: "Formatted at 2026-09-25 13:11:09 on dist-test-slave-chkf"
server_key: "7f5e58ccfa6a6cdcd3dd1ff307b6c6ef"
server_key_iv: "685c586d20d286961d9696b146d524c6"
server_key_version: "encryptionkey@0"
I20260925 13:11:09.753921 21358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:11:09.754598 21373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:09.754807 21358 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:11:09.754904 21358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "a4e611be2d144356800ba704025cd250"
format_stamp: "Formatted at 2026-09-25 13:11:09 on dist-test-slave-chkf"
server_key: "7f5e58ccfa6a6cdcd3dd1ff307b6c6ef"
server_key_iv: "685c586d20d286961d9696b146d524c6"
server_key_version: "encryptionkey@0"
I20260925 13:11:09.754992 21358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:09.767518 21358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:09.769737 21358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:09.769892 21358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:09.773815 21358 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46203
I20260925 13:11:09.773913 21425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46203 every 8 connection(s)
I20260925 13:11:09.774216 21358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:09.775095 21426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:09.777185 21426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250: Bootstrap starting.
I20260925 13:11:09.777684 21426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:09.777932 21426 log.cc:824] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:09.778559 21426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250: No bootstrap required, opened a new log
I20260925 13:11:09.780145 21426 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e611be2d144356800ba704025cd250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46203 } }
I20260925 13:11:09.780284 21426 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:09.780306 21426 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e611be2d144356800ba704025cd250, State: Initialized, Role: FOLLOWER
I20260925 13:11:09.780444 21426 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e611be2d144356800ba704025cd250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46203 } }
I20260925 13:11:09.780526 21426 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:09.780550 21426 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:09.780580 21426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:09.781059 21426 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e611be2d144356800ba704025cd250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46203 } }
I20260925 13:11:09.781142 21426 leader_election.cc:302] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4e611be2d144356800ba704025cd250; no voters:
I20260925 13:11:09.781260 21426 leader_election.cc:288] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:09.781323 21429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:09.781509 21429 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250 [term 1 LEADER]: Becoming Leader. State: Replica: a4e611be2d144356800ba704025cd250, State: Running, Role: LEADER
I20260925 13:11:09.781564 21426 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:09.781598 21429 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e611be2d144356800ba704025cd250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46203 } }
I20260925 13:11:09.781991 21431 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4e611be2d144356800ba704025cd250. Latest consensus state: current_term: 1 leader_uuid: "a4e611be2d144356800ba704025cd250" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e611be2d144356800ba704025cd250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46203 } } }
I20260925 13:11:09.782069 21431 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:09.782114 21430 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4e611be2d144356800ba704025cd250" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e611be2d144356800ba704025cd250" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46203 } } }
I20260925 13:11:09.782171 21430 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:09.782334 21435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:09.782891 21435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:09.783356 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 21358
I20260925 13:11:09.783476 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:09.783659 825 external_mini_cluster.cc:1428] Setting key 557472e6d04046f6f9f735d92d9cecc5
I20260925 13:11:09.785095 21435 catalog_manager.cc:1347] Generated new cluster ID: 2b795cf4908e460d9e5f6e9300c88f7e
I20260925 13:11:09.785144 21435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 13:11:09 dist-test-slave-chkf krb5kdc[21345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 13:11:09.794548 21435 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:09.795238 21435 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:09.795499 21448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:09.784177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56051 (local address 127.0.0.1:46203)
0925 13:11:09.784362 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:09.784372 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:09.784405 (+ 33us) server_negotiation.cc:407] Connection header received
0925 13:11:09.784519 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:09.784526 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:09.784580 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:09.784729 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:09.785577 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:09.786997 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:09.788086 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:09.788671 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:09.788774 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:09.791588 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:09.791592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:09.791592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:09.791604 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:09.793607 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:09.794001 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:09.794004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:09.794004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:09.794113 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:09.794332 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:09.794333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:09.794333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:09.794589 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:09.794759 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:09.794993 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 13:11:09.795284 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":141,"server-negotiator.queue_time_us":126,"thread_start_us":101,"threads_started":1}
I20260925 13:11:09.796368 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:09.800634 21435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4e611be2d144356800ba704025cd250: Generated new TSK 0
I20260925 13:11:09.800783 21435 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:09.804498 21448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:09.797866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44383 (local address 127.0.0.1:46203)
0925 13:11:09.797897 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:09.797898 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:09.797922 (+ 24us) server_negotiation.cc:407] Connection header received
0925 13:11:09.797993 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:09.797994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:09.798012 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:09.798060 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:09.798634 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:09.799372 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:09.800424 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:09.800762 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:09.800820 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:09.801992 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:09.801993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:09.801993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:09.802000 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:09.803491 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:09.803808 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:09.803809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:09.803810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:09.803868 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:09.804039 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:09.804039 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:09.804040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:09.804165 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:09.804262 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:09.804335 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 13:11:09.804412 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 13:11:09.806171 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 21358
2026-09-25T13:11:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:09 dist-test-slave-chkf krb5kdc[21457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:09 dist-test-slave-chkf krb5kdc[21457](info): set up 2 sockets
Sep 25 13:11:09 dist-test-slave-chkf krb5kdc[21457](info): commencing operation
krb5kdc: starting...
W20260925 13:11:10.838663 825 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:10 dist-test-slave-chkf krb5kdc[21457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 13:11:10.953258 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41555
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:11.031991 21470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:11.032233 21470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:11.032274 21470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:11.033422 21470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:11.033493 21470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:11.033516 21470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:11.033537 21470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:11.033569 21470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:11.035070 21470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:11.035341 21470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:11.035564 21470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:11.040406 21475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:11.040431 21476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:11.040431 21478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:11.040794 21470 server_base.cc:1034] running on GCE node
I20260925 13:11:11.041018 21470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:11.041250 21470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:11.042387 21470 hybrid_clock.cc:648] HybridClock initialized: now 1790341871042366 us; error 42 us; skew 500 ppm
Sep 25 13:11:11 dist-test-slave-chkf krb5kdc[21457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341871, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:11.047906 21470 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 13:11:11.048374 21470 webserver.cc:460] Webserver started at http://127.0.0.1:37457/ using document root <none> and password file <none>
I20260925 13:11:11.048597 21470 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:11.048648 21470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:11.048790 21470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:11.049822 21470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "5b1c94906493461d803d6da9434acc42"
format_stamp: "Formatted at 2026-09-25 13:11:11 on dist-test-slave-chkf"
server_key: "e09be822d6bfec43c440e2822a7971ca"
server_key_iv: "c64408319002923eb78fcf09b3e204dd"
server_key_version: "encryptionkey@0"
I20260925 13:11:11.050182 21470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "5b1c94906493461d803d6da9434acc42"
format_stamp: "Formatted at 2026-09-25 13:11:11 on dist-test-slave-chkf"
server_key: "e09be822d6bfec43c440e2822a7971ca"
server_key_iv: "c64408319002923eb78fcf09b3e204dd"
server_key_version: "encryptionkey@0"
I20260925 13:11:11.051594 21470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:11:11.052374 21485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:11.052573 21470 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:11:11.052677 21470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "5b1c94906493461d803d6da9434acc42"
format_stamp: "Formatted at 2026-09-25 13:11:11 on dist-test-slave-chkf"
server_key: "e09be822d6bfec43c440e2822a7971ca"
server_key_iv: "c64408319002923eb78fcf09b3e204dd"
server_key_version: "encryptionkey@0"
I20260925 13:11:11.052750 21470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:11.102917 21470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:11.104892 21470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:11.105050 21470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:11.109153 21470 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41555
I20260925 13:11:11.109228 21537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41555 every 8 connection(s)
I20260925 13:11:11.109572 21470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:11.110430 21538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:11.112792 21538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42: Bootstrap starting.
I20260925 13:11:11.113379 21538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:11.113739 21538 log.cc:824] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:11.114459 21538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42: No bootstrap required, opened a new log
I20260925 13:11:11.116040 21538 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b1c94906493461d803d6da9434acc42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41555 } }
I20260925 13:11:11.116189 21538 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:11.116214 21538 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b1c94906493461d803d6da9434acc42, State: Initialized, Role: FOLLOWER
I20260925 13:11:11.116374 21538 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b1c94906493461d803d6da9434acc42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41555 } }
I20260925 13:11:11.116436 21538 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:11.116477 21538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:11.116526 21538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:11.117069 21538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b1c94906493461d803d6da9434acc42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41555 } }
I20260925 13:11:11.117161 21538 leader_election.cc:302] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b1c94906493461d803d6da9434acc42; no voters:
I20260925 13:11:11.117336 21538 leader_election.cc:288] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:11.117421 21541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:11.117584 21538 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:11.117590 21541 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42 [term 1 LEADER]: Becoming Leader. State: Replica: 5b1c94906493461d803d6da9434acc42, State: Running, Role: LEADER
I20260925 13:11:11.117769 21541 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b1c94906493461d803d6da9434acc42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41555 } }
I20260925 13:11:11.118124 21543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b1c94906493461d803d6da9434acc42. Latest consensus state: current_term: 1 leader_uuid: "5b1c94906493461d803d6da9434acc42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b1c94906493461d803d6da9434acc42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41555 } } }
I20260925 13:11:11.118186 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 21470
I20260925 13:11:11.118312 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:11.118302 21543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:11.118530 825 external_mini_cluster.cc:1428] Setting key cab1c208fc95c669ee6ac8a800535be0
I20260925 13:11:11.119011 21542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b1c94906493461d803d6da9434acc42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b1c94906493461d803d6da9434acc42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41555 } } }
I20260925 13:11:11.119158 21542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:11.119313 21545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:11.119801 21545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:11.121436 21545 catalog_manager.cc:1347] Generated new cluster ID: 616b4ed3e19545c9afecb66b4573f15a
I20260925 13:11:11.121488 21545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:11.124835 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:11.119070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56115 (local address 127.0.0.1:41555)
0925 13:11:11.119261 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:11.119271 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:11.119346 (+ 75us) server_negotiation.cc:407] Connection header received
0925 13:11:11.119465 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:11.119473 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:11.119531 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:11.119678 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:11.119948 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:11.121202 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:11.122348 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:11.123028 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:11.123486 (+ 458us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:11.123935 (+ 449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:11.123939 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:11.123939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:11.124160 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:11.124198 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:11.124214 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:11.124277 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:11.124544 (+ 267us) server_negotiation.cc:299] Negotiation successful
0925 13:11:11.124613 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":73,"threads_started":1}
I20260925 13:11:11.125801 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:11.134310 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:11.127172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42767 (local address 127.0.0.1:41555)
0925 13:11:11.127207 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:11.127208 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:11.130903 (+ 3695us) server_negotiation.cc:407] Connection header received
0925 13:11:11.130991 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:11.130992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:11.131003 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:11.131063 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:11.131191 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:11.131935 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:11.132988 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:11.133278 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:11.133338 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:11.134050 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:11.134051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:11.134052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:11.134057 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:11.134072 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:11.134076 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:11.134129 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:11.134205 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 13:11:11.134232 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:11:11.144677 21545 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:11.145213 21545 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:11.148757 21545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b1c94906493461d803d6da9434acc42: Generated new TSK 0
I20260925 13:11:11.148924 21545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:11.153688 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 21470
2026-09-25T13:11:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1347 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:11 dist-test-slave-chkf krb5kdc[21569](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:11 dist-test-slave-chkf krb5kdc[21569](info): set up 2 sockets
Sep 25 13:11:11 dist-test-slave-chkf krb5kdc[21569](info): commencing operation
krb5kdc: starting...
W20260925 13:11:12.190681 825 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:12 dist-test-slave-chkf krb5kdc[21569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 13:11:12.303053 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45207
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:12.382606 21582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:12.382859 21582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:12.382900 21582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:12.384027 21582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:12.384095 21582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:12.384123 21582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:12.384155 21582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:12.384181 21582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:12.385615 21582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:12.385879 21582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:12.386103 21582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:12.390829 21587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:12.390810 21588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:12.390889 21582 server_base.cc:1034] running on GCE node
W20260925 13:11:12.390808 21590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:12.391322 21582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:12.391564 21582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:12.392704 21582 hybrid_clock.cc:648] HybridClock initialized: now 1790341872392693 us; error 29 us; skew 500 ppm
Sep 25 13:11:12 dist-test-slave-chkf krb5kdc[21569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341872, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:12.395073 21582 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 13:11:12.395490 21582 webserver.cc:460] Webserver started at http://127.0.0.1:35381/ using document root <none> and password file <none>
I20260925 13:11:12.395727 21582 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:12.395774 21582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:12.395921 21582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:12.396970 21582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "d7fdfaf7b0bd44f99ac389c2ad2fcfee"
format_stamp: "Formatted at 2026-09-25 13:11:12 on dist-test-slave-chkf"
server_key: "c4c218e48356d6e60fc92cf5e1ced57b"
server_key_iv: "1f409c81dac175ba57eec27bc2e2c0c6"
server_key_version: "encryptionkey@0"
I20260925 13:11:12.397358 21582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "d7fdfaf7b0bd44f99ac389c2ad2fcfee"
format_stamp: "Formatted at 2026-09-25 13:11:12 on dist-test-slave-chkf"
server_key: "c4c218e48356d6e60fc92cf5e1ced57b"
server_key_iv: "1f409c81dac175ba57eec27bc2e2c0c6"
server_key_version: "encryptionkey@0"
I20260925 13:11:12.398674 21582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:11:12.399432 21597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:12.399628 21582 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:11:12.399726 21582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "d7fdfaf7b0bd44f99ac389c2ad2fcfee"
format_stamp: "Formatted at 2026-09-25 13:11:12 on dist-test-slave-chkf"
server_key: "c4c218e48356d6e60fc92cf5e1ced57b"
server_key_iv: "1f409c81dac175ba57eec27bc2e2c0c6"
server_key_version: "encryptionkey@0"
I20260925 13:11:12.399813 21582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:12.403033 21582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:12.404857 21582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:12.405010 21582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:12.408944 21582 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45207
I20260925 13:11:12.408973 21649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45207 every 8 connection(s)
I20260925 13:11:12.409310 21582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:12.410231 21650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:12.412402 21650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee: Bootstrap starting.
I20260925 13:11:12.412954 21650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:12.413230 21650 log.cc:824] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:12.413908 21650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee: No bootstrap required, opened a new log
I20260925 13:11:12.415494 21650 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fdfaf7b0bd44f99ac389c2ad2fcfee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45207 } }
I20260925 13:11:12.415623 21650 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:12.415645 21650 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7fdfaf7b0bd44f99ac389c2ad2fcfee, State: Initialized, Role: FOLLOWER
I20260925 13:11:12.415841 21650 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fdfaf7b0bd44f99ac389c2ad2fcfee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45207 } }
I20260925 13:11:12.415901 21650 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:12.415923 21650 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:12.415985 21650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:12.416483 21650 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fdfaf7b0bd44f99ac389c2ad2fcfee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45207 } }
I20260925 13:11:12.416590 21650 leader_election.cc:302] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7fdfaf7b0bd44f99ac389c2ad2fcfee; no voters:
I20260925 13:11:12.416767 21650 leader_election.cc:288] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:12.416846 21653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:12.417021 21653 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee [term 1 LEADER]: Becoming Leader. State: Replica: d7fdfaf7b0bd44f99ac389c2ad2fcfee, State: Running, Role: LEADER
I20260925 13:11:12.417019 21650 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:12.417124 21653 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fdfaf7b0bd44f99ac389c2ad2fcfee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45207 } }
I20260925 13:11:12.417414 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 21582
I20260925 13:11:12.417536 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:12.417727 825 external_mini_cluster.cc:1428] Setting key eee832cea97cfccc25e306dfcbe4ff51
I20260925 13:11:12.418533 21654 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7fdfaf7b0bd44f99ac389c2ad2fcfee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fdfaf7b0bd44f99ac389c2ad2fcfee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45207 } } }
I20260925 13:11:12.418793 21654 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:12.418531 21655 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7fdfaf7b0bd44f99ac389c2ad2fcfee. Latest consensus state: current_term: 1 leader_uuid: "d7fdfaf7b0bd44f99ac389c2ad2fcfee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fdfaf7b0bd44f99ac389c2ad2fcfee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45207 } } }
I20260925 13:11:12.418991 21655 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee [sys.catalog]: This master's current role is: LEADER
W20260925 13:11:12.419088 21669 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:11:12.419145 21669 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:11:12.419207 21672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:12.419446 21672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:12.420324 21666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:12.418193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41205 (local address 127.0.0.1:45207)
0925 13:11:12.418906 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:12.418915 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:12.418951 (+ 36us) server_negotiation.cc:407] Connection header received
0925 13:11:12.419068 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:12.419077 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:12.419108 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:12.419278 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:12.419877 (+ 599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:12.419880 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:12.419880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:12.419885 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:12.419927 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:12.419941 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:12.420009 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:12.420099 (+ 90us) server_negotiation.cc:299] Negotiation successful
0925 13:11:12.420144 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":74,"threads_started":1}
I20260925 13:11:12.421239 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:12.421497 21672 catalog_manager.cc:1347] Generated new cluster ID: 2476d515e1ec4f35b00b30b84b8c3272
I20260925 13:11:12.421542 21672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:12.423758 21666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:12.422659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57443 (local address 127.0.0.1:45207)
0925 13:11:12.422690 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:12.422692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:12.422879 (+ 187us) server_negotiation.cc:407] Connection header received
0925 13:11:12.422920 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:12.422921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:12.422926 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:12.422985 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:12.423546 (+ 561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:12.423547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:12.423547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:12.423551 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:12.423565 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:12.423569 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:12.423638 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:12.423687 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 13:11:12.423700 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 13:11:12.428301 21672 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:12.428436 21672 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:12.432394 21672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7fdfaf7b0bd44f99ac389c2ad2fcfee: Generated new TSK 0
I20260925 13:11:12.432564 21672 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:12.436486 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 21582
2026-09-25T13:11:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1283 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:12 dist-test-slave-chkf krb5kdc[21681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:12 dist-test-slave-chkf krb5kdc[21681](info): set up 2 sockets
Sep 25 13:11:12 dist-test-slave-chkf krb5kdc[21681](info): commencing operation
krb5kdc: starting...
W20260925 13:11:13.478695 825 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:13 dist-test-slave-chkf krb5kdc[21681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 13:11:13.589562 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41857
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:13.668141 21694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:13.668376 21694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:13.668416 21694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:13.669556 21694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:13.669623 21694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:13.669649 21694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:13.669679 21694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:13.669701 21694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:13.671203 21694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:13.671468 21694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:13.671686 21694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:13.676486 21702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:13.676499 21700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:13.676694 21694 server_base.cc:1034] running on GCE node
W20260925 13:11:13.676496 21699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:13.676991 21694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:13.677302 21694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:13.678464 21694 hybrid_clock.cc:648] HybridClock initialized: now 1790341873678448 us; error 25 us; skew 500 ppm
Sep 25 13:11:13 dist-test-slave-chkf krb5kdc[21681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341873, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:13.680954 21694 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 13:11:13.681378 21694 webserver.cc:460] Webserver started at http://127.0.0.1:38809/ using document root <none> and password file <none>
I20260925 13:11:13.681602 21694 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:13.681668 21694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:13.681800 21694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:13.682901 21694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "5c8ec81eeb7a47668f7528da3c3d7d01"
format_stamp: "Formatted at 2026-09-25 13:11:13 on dist-test-slave-chkf"
server_key: "49db6af2b050557989968ef22ab65aad"
server_key_iv: "475406f1329dec62ef1da01a686f791b"
server_key_version: "encryptionkey@0"
I20260925 13:11:13.683260 21694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "5c8ec81eeb7a47668f7528da3c3d7d01"
format_stamp: "Formatted at 2026-09-25 13:11:13 on dist-test-slave-chkf"
server_key: "49db6af2b050557989968ef22ab65aad"
server_key_iv: "475406f1329dec62ef1da01a686f791b"
server_key_version: "encryptionkey@0"
I20260925 13:11:13.684587 21694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 13:11:13.685279 21709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:13.685441 21694 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:13.685551 21694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "5c8ec81eeb7a47668f7528da3c3d7d01"
format_stamp: "Formatted at 2026-09-25 13:11:13 on dist-test-slave-chkf"
server_key: "49db6af2b050557989968ef22ab65aad"
server_key_iv: "475406f1329dec62ef1da01a686f791b"
server_key_version: "encryptionkey@0"
I20260925 13:11:13.685622 21694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:13.696477 21694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:13.698753 21694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:13.698915 21694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:13.702502 21694 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41857
I20260925 13:11:13.702534 21761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41857 every 8 connection(s)
I20260925 13:11:13.702912 21694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:13.703750 21762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:13.704118 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 21694
I20260925 13:11:13.704236 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:13.704411 825 external_mini_cluster.cc:1428] Setting key 63f140d89a7a7f53a3bca4d8009c7087
I20260925 13:11:13.706699 21762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01: Bootstrap starting.
I20260925 13:11:13.707393 21762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:13.707690 21762 log.cc:824] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:13.708380 21762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01: No bootstrap required, opened a new log
I20260925 13:11:13.710459 21762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c8ec81eeb7a47668f7528da3c3d7d01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41857 } }
I20260925 13:11:13.710608 21762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:13.710655 21762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c8ec81eeb7a47668f7528da3c3d7d01, State: Initialized, Role: FOLLOWER
I20260925 13:11:13.710836 21762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c8ec81eeb7a47668f7528da3c3d7d01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41857 } }
I20260925 13:11:13.710932 21762 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:13.710974 21762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:13.711021 21762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01 [term 0 FOLLOWER]: Advancing to term 1
Sep 25 13:11:13 dist-test-slave-chkf krb5kdc[21681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 13:11:13.711611 21762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c8ec81eeb7a47668f7528da3c3d7d01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41857 } }
I20260925 13:11:13.711717 21762 leader_election.cc:302] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c8ec81eeb7a47668f7528da3c3d7d01; no voters:
I20260925 13:11:13.711884 21762 leader_election.cc:288] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:13.711958 21767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:13.712112 21767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01 [term 1 LEADER]: Becoming Leader. State: Replica: 5c8ec81eeb7a47668f7528da3c3d7d01, State: Running, Role: LEADER
I20260925 13:11:13.712194 21762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:13.712255 21767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c8ec81eeb7a47668f7528da3c3d7d01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41857 } }
I20260925 13:11:13.712551 21769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c8ec81eeb7a47668f7528da3c3d7d01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c8ec81eeb7a47668f7528da3c3d7d01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41857 } } }
I20260925 13:11:13.712769 21768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c8ec81eeb7a47668f7528da3c3d7d01. Latest consensus state: current_term: 1 leader_uuid: "5c8ec81eeb7a47668f7528da3c3d7d01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c8ec81eeb7a47668f7528da3c3d7d01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41857 } } }
I20260925 13:11:13.713251 21768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:13.712711 21769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01 [sys.catalog]: This master's current role is: LEADER
W20260925 13:11:13.714128 21783 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:11:13.714186 21783 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:11:13.714236 21780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:13.714401 21780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:13.715610 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:13.704902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:41463 (local address 127.0.0.1:41857)
0925 13:11:13.705090 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:13.705100 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:13.705133 (+ 33us) server_negotiation.cc:407] Connection header received
0925 13:11:13.705661 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:13.705669 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:13.705729 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:13.705877 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:13.706430 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:13.707801 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:13.708666 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:13.709264 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:13.709377 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:13.711638 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:13.711642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:13.711642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:13.711653 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:13.713868 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:13.714335 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:13.714339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:13.714339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:13.714442 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:13.714686 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:13.714687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:13.714688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:13.715011 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:13.715166 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:13.715294 (+ 128us) server_negotiation.cc:299] Negotiation successful
0925 13:11:13.715395 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260925 13:11:13.716217 21780 catalog_manager.cc:1347] Generated new cluster ID: 46c6d5b2a9e54fdebb29d0993ee027da
I20260925 13:11:13.716270 21780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:13.716526 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:13.725730 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:13.717964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:59425 (local address 127.0.0.1:41857)
0925 13:11:13.717999 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:13.718001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:13.718114 (+ 113us) server_negotiation.cc:407] Connection header received
0925 13:11:13.718205 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:13.718206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:13.718215 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:13.718254 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:13.718969 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:13.719986 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:13.721038 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:13.721376 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:13.721418 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:13.722924 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:13.722925 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:13.722925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:13.722932 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:13.724414 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:13.724851 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:13.724852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:13.724853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:13.724904 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:13.725132 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:13.725133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:13.725133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:13.725262 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:13.725337 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:13.725575 (+ 238us) server_negotiation.cc:299] Negotiation successful
0925 13:11:13.725639 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 13:11:13.743614 21780 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:13.744120 21780 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:13.751091 21780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c8ec81eeb7a47668f7528da3c3d7d01: Generated new TSK 0
I20260925 13:11:13.751292 21780 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:13.756405 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 21694
W20260925 13:11:13.760603 21679 connection.cc:535] client connection to 127.0.0.1:41857 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41857): Connection reset by peer (error 104)
2026-09-25T13:11:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1319 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:13 dist-test-slave-chkf krb5kdc[21793](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:13 dist-test-slave-chkf krb5kdc[21793](info): set up 2 sockets
Sep 25 13:11:13 dist-test-slave-chkf krb5kdc[21793](info): commencing operation
krb5kdc: starting...
W20260925 13:11:14.782920 825 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:14 dist-test-slave-chkf krb5kdc[21793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 13:11:14.893354 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41945
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:14.973977 21806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:14.974159 21806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:14.974180 21806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:14.975383 21806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:14.975423 21806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:14.975438 21806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:14.975451 21806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:14.975498 21806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:14.976897 21806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:14.977116 21806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:14.977279 21806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:14.982170 21814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:14.982239 21811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:14.982200 21812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:14.982271 21806 server_base.cc:1034] running on GCE node
I20260925 13:11:14.982589 21806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:14.982920 21806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:14.984061 21806 hybrid_clock.cc:648] HybridClock initialized: now 1790341874984038 us; error 40 us; skew 500 ppm
Sep 25 13:11:14 dist-test-slave-chkf krb5kdc[21793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341874, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:14.986537 21806 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 13:11:14.987015 21806 webserver.cc:460] Webserver started at http://127.0.0.1:32797/ using document root <none> and password file <none>
I20260925 13:11:14.987246 21806 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:14.987313 21806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:14.987447 21806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:14.988497 21806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "00ce04a144584f76b7370d60c9fed6ad"
format_stamp: "Formatted at 2026-09-25 13:11:14 on dist-test-slave-chkf"
server_key: "11916c191d754461225301de7c6eb71c"
server_key_iv: "6ea831ece6218fc2fcfc322b24bc50d6"
server_key_version: "encryptionkey@0"
I20260925 13:11:14.988857 21806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "00ce04a144584f76b7370d60c9fed6ad"
format_stamp: "Formatted at 2026-09-25 13:11:14 on dist-test-slave-chkf"
server_key: "11916c191d754461225301de7c6eb71c"
server_key_iv: "6ea831ece6218fc2fcfc322b24bc50d6"
server_key_version: "encryptionkey@0"
I20260925 13:11:14.990127 21806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:11:14.990876 21821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:14.991051 21806 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:11:14.991147 21806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "00ce04a144584f76b7370d60c9fed6ad"
format_stamp: "Formatted at 2026-09-25 13:11:14 on dist-test-slave-chkf"
server_key: "11916c191d754461225301de7c6eb71c"
server_key_iv: "6ea831ece6218fc2fcfc322b24bc50d6"
server_key_version: "encryptionkey@0"
I20260925 13:11:14.991231 21806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:15.006404 21806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:15.008378 21806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:15.008543 21806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:15.012455 21806 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41945
I20260925 13:11:15.012492 21873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41945 every 8 connection(s)
I20260925 13:11:15.012858 21806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:15.013682 21874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:15.015909 21874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad: Bootstrap starting.
I20260925 13:11:15.016419 21874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:15.016649 21874 log.cc:824] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:15.017272 21874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad: No bootstrap required, opened a new log
I20260925 13:11:15.017709 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 21806
I20260925 13:11:15.017846 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:15.018030 825 external_mini_cluster.cc:1428] Setting key 3bbb4633375f6e4b08792bf456449d36
I20260925 13:11:15.019599 21874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ce04a144584f76b7370d60c9fed6ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41945 } }
I20260925 13:11:15.019797 21874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:15.019833 21874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ce04a144584f76b7370d60c9fed6ad, State: Initialized, Role: FOLLOWER
I20260925 13:11:15.019958 21874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ce04a144584f76b7370d60c9fed6ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41945 } }
I20260925 13:11:15.020044 21874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:15.020079 21874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:15.020162 21874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:15.020696 21874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ce04a144584f76b7370d60c9fed6ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41945 } }
I20260925 13:11:15.020782 21874 leader_election.cc:302] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00ce04a144584f76b7370d60c9fed6ad; no voters:
I20260925 13:11:15.020987 21874 leader_election.cc:288] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:15.021015 21879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:15.021121 21879 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad [term 1 LEADER]: Becoming Leader. State: Replica: 00ce04a144584f76b7370d60c9fed6ad, State: Running, Role: LEADER
I20260925 13:11:15.021250 21879 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ce04a144584f76b7370d60c9fed6ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41945 } }
I20260925 13:11:15.021248 21874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:15.021988 21881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00ce04a144584f76b7370d60c9fed6ad. Latest consensus state: current_term: 1 leader_uuid: "00ce04a144584f76b7370d60c9fed6ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ce04a144584f76b7370d60c9fed6ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41945 } } }
I20260925 13:11:15.022066 21881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:15.022267 21880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00ce04a144584f76b7370d60c9fed6ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ce04a144584f76b7370d60c9fed6ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41945 } } }
I20260925 13:11:15.022344 21880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:15.022468 21889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:15.022969 21889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:15.024566 21889 catalog_manager.cc:1347] Generated new cluster ID: 274f54238402410e93fe78d96d80a2c0
I20260925 13:11:15.024618 21889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:15.024931 21878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:15.018487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:33943 (local address 127.0.0.1:41945)
0925 13:11:15.018688 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:15.018697 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:15.018761 (+ 64us) server_negotiation.cc:407] Connection header received
0925 13:11:15.019812 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:15.019822 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:15.019887 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:15.020033 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:15.020323 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:15.021574 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:15.022801 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:15.023393 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:15.023503 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:15.024238 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:15.024242 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:15.024242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:15.024448 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:15.024483 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:15.024498 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:15.024569 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:15.024721 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 13:11:15.024748 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260925 13:11:15.027726 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:15.032728 21878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:15.029257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:49125 (local address 127.0.0.1:41945)
0925 13:11:15.029293 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:15.029294 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:15.029495 (+ 201us) server_negotiation.cc:407] Connection header received
0925 13:11:15.029558 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:15.029558 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:15.029567 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:15.029611 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:15.029740 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:15.030419 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:15.031436 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:15.031730 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:15.031786 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:15.032491 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:15.032492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:15.032493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:15.032497 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:15.032510 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:15.032515 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:15.032566 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:15.032652 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 13:11:15.032661 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 13:11:15.034773 21889 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:15.035419 21889 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:15.041491 21889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00ce04a144584f76b7370d60c9fed6ad: Generated new TSK 0
I20260925 13:11:15.041657 21889 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:15.047431 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 21806
W20260925 13:11:15.051694 21791 connection.cc:535] client connection to 127.0.0.1:41945 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41945): Connection reset by peer (error 104)
2026-09-25T13:11:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1291 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:15 dist-test-slave-chkf krb5kdc[21905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:15 dist-test-slave-chkf krb5kdc[21905](info): set up 2 sockets
Sep 25 13:11:15 dist-test-slave-chkf krb5kdc[21905](info): commencing operation
krb5kdc: starting...
W20260925 13:11:16.070788 825 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:16 dist-test-slave-chkf krb5kdc[21905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 13:11:16.185377 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45071
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:16.265141 21918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:16.265379 21918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:16.265416 21918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:16.266537 21918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:16.266605 21918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:16.266631 21918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:16.266662 21918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:16.266686 21918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:16.268155 21918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:16.268338 21918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:16.268499 21918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:16.273113 21924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:16.273113 21926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:16.273157 21918 server_base.cc:1034] running on GCE node
W20260925 13:11:16.273168 21923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:16.273595 21918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:16.273844 21918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:16.274983 21918 hybrid_clock.cc:648] HybridClock initialized: now 1790341876274970 us; error 29 us; skew 500 ppm
Sep 25 13:11:16 dist-test-slave-chkf krb5kdc[21905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341876, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:16.281708 21918 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 13:11:16.282166 21918 webserver.cc:460] Webserver started at http://127.0.0.1:37851/ using document root <none> and password file <none>
I20260925 13:11:16.282346 21918 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:16.282387 21918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:16.282474 21918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:16.283548 21918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "c6e33d9e25634e928c8570cbae946d42"
format_stamp: "Formatted at 2026-09-25 13:11:16 on dist-test-slave-chkf"
server_key: "8b1a9f3b46e4fc52e1fd5cca15fa7f1b"
server_key_iv: "1c3694be1d331048527ee679a2ffb2be"
server_key_version: "encryptionkey@0"
I20260925 13:11:16.283852 21918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "c6e33d9e25634e928c8570cbae946d42"
format_stamp: "Formatted at 2026-09-25 13:11:16 on dist-test-slave-chkf"
server_key: "8b1a9f3b46e4fc52e1fd5cca15fa7f1b"
server_key_iv: "1c3694be1d331048527ee679a2ffb2be"
server_key_version: "encryptionkey@0"
I20260925 13:11:16.285056 21918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 13:11:16.285696 21933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:16.285851 21918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:11:16.285931 21918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "c6e33d9e25634e928c8570cbae946d42"
format_stamp: "Formatted at 2026-09-25 13:11:16 on dist-test-slave-chkf"
server_key: "8b1a9f3b46e4fc52e1fd5cca15fa7f1b"
server_key_iv: "1c3694be1d331048527ee679a2ffb2be"
server_key_version: "encryptionkey@0"
I20260925 13:11:16.285983 21918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:16.289209 21918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:16.291033 21918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:16.291136 21918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:16.294638 21918 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45071
I20260925 13:11:16.294727 21985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45071 every 8 connection(s)
I20260925 13:11:16.295027 21918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:16.295966 21986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:16.298027 21986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42: Bootstrap starting.
I20260925 13:11:16.298527 21986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:16.298825 21986 log.cc:824] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:16.299448 21986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42: No bootstrap required, opened a new log
I20260925 13:11:16.299820 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 21918
I20260925 13:11:16.299948 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:16.300132 825 external_mini_cluster.cc:1428] Setting key a130b5116cced678cbd776e03fd05531
I20260925 13:11:16.301816 21986 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6e33d9e25634e928c8570cbae946d42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45071 } }
I20260925 13:11:16.301994 21986 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:16.302038 21986 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6e33d9e25634e928c8570cbae946d42, State: Initialized, Role: FOLLOWER
I20260925 13:11:16.302187 21986 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6e33d9e25634e928c8570cbae946d42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45071 } }
I20260925 13:11:16.302296 21986 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:16.302338 21986 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:16.302371 21986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:16.302846 21990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:16.300607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:55299 (local address 127.0.0.1:45071)
0925 13:11:16.300763 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:16.300778 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:16.300801 (+ 23us) server_negotiation.cc:407] Connection header received
0925 13:11:16.301337 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:16.301343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:16.301366 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:16.301471 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:16.302110 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:16.302114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:16.302114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:16.302118 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:16.302190 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:16.302206 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:16.302284 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:16.302565 (+ 281us) server_negotiation.cc:299] Negotiation successful
0925 13:11:16.302613 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260925 13:11:16.303414 21986 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6e33d9e25634e928c8570cbae946d42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45071 } }
I20260925 13:11:16.303521 21986 leader_election.cc:302] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6e33d9e25634e928c8570cbae946d42; no voters:
I20260925 13:11:16.303718 21986 leader_election.cc:288] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:16.303725 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:16.303741 21991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:16.303857 21991 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42 [term 1 LEADER]: Becoming Leader. State: Replica: c6e33d9e25634e928c8570cbae946d42, State: Running, Role: LEADER
I20260925 13:11:16.303978 21986 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:16.303992 21991 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6e33d9e25634e928c8570cbae946d42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45071 } }
I20260925 13:11:16.304425 21993 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6e33d9e25634e928c8570cbae946d42. Latest consensus state: current_term: 1 leader_uuid: "c6e33d9e25634e928c8570cbae946d42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6e33d9e25634e928c8570cbae946d42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45071 } } }
I20260925 13:11:16.304517 21993 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:16.304750 21992 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6e33d9e25634e928c8570cbae946d42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6e33d9e25634e928c8570cbae946d42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45071 } } }
I20260925 13:11:16.305063 21992 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:16.305294 21999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:16.305790 21999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:16.307307 21999 catalog_manager.cc:1347] Generated new cluster ID: 9c80fca329a54ad097e24be142fe0623
I20260925 13:11:16.307353 21999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:16.311142 21990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:16.306524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:45233 (local address 127.0.0.1:45071)
0925 13:11:16.306559 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:16.306561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:16.306822 (+ 261us) server_negotiation.cc:407] Connection header received
0925 13:11:16.306885 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:16.306886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:16.306891 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:16.306952 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:16.310911 (+ 3959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:16.310912 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:16.310913 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:16.310917 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:16.310932 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:16.310937 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:16.311004 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:16.311067 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 13:11:16.311080 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:11:16.313994 21999 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:16.314141 21999 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:16.317883 21999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6e33d9e25634e928c8570cbae946d42: Generated new TSK 0
I20260925 13:11:16.318068 21999 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:16.320791 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 21918
2026-09-25T13:11:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1272 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:16 dist-test-slave-chkf krb5kdc[22017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:16 dist-test-slave-chkf krb5kdc[22017](info): set up 2 sockets
Sep 25 13:11:16 dist-test-slave-chkf krb5kdc[22017](info): commencing operation
krb5kdc: starting...
W20260925 13:11:17.330883 825 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:17 dist-test-slave-chkf krb5kdc[22017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 13:11:17.450342 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37075
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:17.528214 22030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:17.528399 22030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:17.528422 22030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:17.529471 22030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:17.529505 22030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:17.529518 22030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:17.529531 22030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:17.529543 22030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:17.531005 22030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37075
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:17.531275 22030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:17.531486 22030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:17.536140 22036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:17.536106 22035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:17.536114 22038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:17.536195 22030 server_base.cc:1034] running on GCE node
I20260925 13:11:17.536509 22030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:17.536808 22030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:17.537945 22030 hybrid_clock.cc:648] HybridClock initialized: now 1790341877537934 us; error 25 us; skew 500 ppm
Sep 25 13:11:17 dist-test-slave-chkf krb5kdc[22017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341877, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:17.540391 22030 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 13:11:17.540853 22030 webserver.cc:460] Webserver started at http://127.0.0.1:43891/ using document root <none> and password file <none>
I20260925 13:11:17.541085 22030 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:17.541129 22030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:17.541294 22030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:17.542299 22030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "176f08193f1c4973b1e34f98532f691e"
format_stamp: "Formatted at 2026-09-25 13:11:17 on dist-test-slave-chkf"
server_key: "f01ddaf89048b53fc1e412bd53d2cb3d"
server_key_iv: "35ca0dbc294e62b4fd31575625cc4bbf"
server_key_version: "encryptionkey@0"
I20260925 13:11:17.542663 22030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "176f08193f1c4973b1e34f98532f691e"
format_stamp: "Formatted at 2026-09-25 13:11:17 on dist-test-slave-chkf"
server_key: "f01ddaf89048b53fc1e412bd53d2cb3d"
server_key_iv: "35ca0dbc294e62b4fd31575625cc4bbf"
server_key_version: "encryptionkey@0"
I20260925 13:11:17.543972 22030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:11:17.544670 22045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:17.544822 22030 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:11:17.544906 22030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "176f08193f1c4973b1e34f98532f691e"
format_stamp: "Formatted at 2026-09-25 13:11:17 on dist-test-slave-chkf"
server_key: "f01ddaf89048b53fc1e412bd53d2cb3d"
server_key_iv: "35ca0dbc294e62b4fd31575625cc4bbf"
server_key_version: "encryptionkey@0"
I20260925 13:11:17.544957 22030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:17.556056 22030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:17.557653 22030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:17.557756 22030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:17.561628 22030 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37075
I20260925 13:11:17.561658 22097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37075 every 8 connection(s)
I20260925 13:11:17.562018 22030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:17.562925 22098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:17.565074 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 22030
I20260925 13:11:17.565157 22098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e: Bootstrap starting.
I20260925 13:11:17.565201 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:17.565392 825 external_mini_cluster.cc:1428] Setting key da37f0d2ba629f15ebce389779f8e117
I20260925 13:11:17.565878 22098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:17.566195 22098 log.cc:824] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:17.566984 22098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e: No bootstrap required, opened a new log
I20260925 13:11:17.569137 22098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176f08193f1c4973b1e34f98532f691e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37075 } }
I20260925 13:11:17.569303 22098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:17.569334 22098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 176f08193f1c4973b1e34f98532f691e, State: Initialized, Role: FOLLOWER
I20260925 13:11:17.569440 22098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176f08193f1c4973b1e34f98532f691e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37075 } }
I20260925 13:11:17.569507 22098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:17.569535 22098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:17.569583 22098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:17.570107 22098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176f08193f1c4973b1e34f98532f691e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37075 } }
I20260925 13:11:17.570226 22098 leader_election.cc:302] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 176f08193f1c4973b1e34f98532f691e; no voters:
I20260925 13:11:17.570405 22098 leader_election.cc:288] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:17.570446 22103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:17.570583 22103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e [term 1 LEADER]: Becoming Leader. State: Replica: 176f08193f1c4973b1e34f98532f691e, State: Running, Role: LEADER
I20260925 13:11:17.570696 22098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:17.570677 22103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176f08193f1c4973b1e34f98532f691e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37075 } }
I20260925 13:11:17.571156 22104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 176f08193f1c4973b1e34f98532f691e. Latest consensus state: current_term: 1 leader_uuid: "176f08193f1c4973b1e34f98532f691e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176f08193f1c4973b1e34f98532f691e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37075 } } }
I20260925 13:11:17.571406 22104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:17.571328 22105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "176f08193f1c4973b1e34f98532f691e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176f08193f1c4973b1e34f98532f691e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37075 } } }
I20260925 13:11:17.571658 22105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:17.571830 22112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:17.572368 22112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 13:11:17 dist-test-slave-chkf krb5kdc[22017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 13:11:17.574506 22112 catalog_manager.cc:1347] Generated new cluster ID: 62b840a63d704396a8b684ce7acfaea6
I20260925 13:11:17.574556 22112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:17.576938 22101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:17.565917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:46791 (local address 127.0.0.1:37075)
0925 13:11:17.566071 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:17.566080 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:17.566116 (+ 36us) server_negotiation.cc:407] Connection header received
0925 13:11:17.566746 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:17.566756 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:17.566820 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:17.566943 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:17.567704 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:17.568550 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:17.569616 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:17.570040 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:17.570142 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:17.573333 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:17.573337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:17.573337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:17.573349 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:17.575281 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:17.575764 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:17.575767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:17.575767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:17.575863 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:17.576096 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:17.576096 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:17.576097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:17.576359 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:17.576497 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:17.576650 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 13:11:17.576744 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
I20260925 13:11:17.577939 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:17.586269 22101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:17.579290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:58713 (local address 127.0.0.1:37075)
0925 13:11:17.579319 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:17.579320 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:17.579441 (+ 121us) server_negotiation.cc:407] Connection header received
0925 13:11:17.579494 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:17.579495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:17.579504 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:17.579556 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:17.580157 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:17.580879 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:17.581869 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:17.582179 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:17.582234 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:17.583567 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:17.583568 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:17.583568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:17.583575 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:17.585048 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:17.585467 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:17.585468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:17.585468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:17.585517 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:17.585778 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:17.585779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:17.585779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:17.585883 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:17.585981 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:17.586135 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 13:11:17.586190 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 13:11:17.592133 22112 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:17.592639 22112 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:17.597484 22112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 176f08193f1c4973b1e34f98532f691e: Generated new TSK 0
I20260925 13:11:17.597682 22112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:17.602880 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 22030
W20260925 13:11:17.607362 22015 connection.cc:535] client connection to 127.0.0.1:37075 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37075): Connection reset by peer (error 104)
2026-09-25T13:11:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1282 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:17 dist-test-slave-chkf krb5kdc[22129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:17 dist-test-slave-chkf krb5kdc[22129](info): set up 2 sockets
Sep 25 13:11:17 dist-test-slave-chkf krb5kdc[22129](info): commencing operation
krb5kdc: starting...
W20260925 13:11:18.626888 825 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:18 dist-test-slave-chkf krb5kdc[22129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 13:11:18.737658 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45007
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:18.817103 22142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:18.817297 22142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:18.817319 22142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:18.818435 22142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:18.818476 22142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:18.818490 22142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:18.818503 22142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:18.818552 22142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:18.819977 22142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45007
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:18.820205 22142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:18.820441 22142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:18.825305 22147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:18.825362 22148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:18.825418 22150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:18.825390 22142 server_base.cc:1034] running on GCE node
I20260925 13:11:18.825841 22142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:18.826115 22142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:18.827250 22142 hybrid_clock.cc:648] HybridClock initialized: now 1790341878827240 us; error 29 us; skew 500 ppm
Sep 25 13:11:18 dist-test-slave-chkf krb5kdc[22129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341878, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:18.829576 22142 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 13:11:18.830034 22142 webserver.cc:460] Webserver started at http://127.0.0.1:41609/ using document root <none> and password file <none>
I20260925 13:11:18.830255 22142 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:18.830300 22142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:18.830440 22142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:18.831509 22142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "5a4791316ded4254945739a555c6b533"
format_stamp: "Formatted at 2026-09-25 13:11:18 on dist-test-slave-chkf"
server_key: "939341bd9ce682b1a63a9f423203a04d"
server_key_iv: "c9b77c16d419b70bdfb60c89c387d460"
server_key_version: "encryptionkey@0"
I20260925 13:11:18.831852 22142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "5a4791316ded4254945739a555c6b533"
format_stamp: "Formatted at 2026-09-25 13:11:18 on dist-test-slave-chkf"
server_key: "939341bd9ce682b1a63a9f423203a04d"
server_key_iv: "c9b77c16d419b70bdfb60c89c387d460"
server_key_version: "encryptionkey@0"
I20260925 13:11:18.833117 22142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 13:11:18.833889 22157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:18.834110 22142 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:11:18.834209 22142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "5a4791316ded4254945739a555c6b533"
format_stamp: "Formatted at 2026-09-25 13:11:18 on dist-test-slave-chkf"
server_key: "939341bd9ce682b1a63a9f423203a04d"
server_key_iv: "c9b77c16d419b70bdfb60c89c387d460"
server_key_version: "encryptionkey@0"
I20260925 13:11:18.834297 22142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:18.849152 22142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:18.850943 22142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:18.851083 22142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:18.855026 22142 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45007
I20260925 13:11:18.855093 22209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45007 every 8 connection(s)
I20260925 13:11:18.855399 22142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:18.856317 22210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:18.858443 22210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533: Bootstrap starting.
I20260925 13:11:18.859032 22210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:18.859269 22210 log.cc:824] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:18.859874 22210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533: No bootstrap required, opened a new log
I20260925 13:11:18.861366 22210 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4791316ded4254945739a555c6b533" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45007 } }
I20260925 13:11:18.861502 22210 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:18.861526 22210 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a4791316ded4254945739a555c6b533, State: Initialized, Role: FOLLOWER
I20260925 13:11:18.861634 22210 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4791316ded4254945739a555c6b533" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45007 } }
I20260925 13:11:18.861687 22210 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:18.861708 22210 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:18.861737 22210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:18.862192 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 22142
I20260925 13:11:18.862314 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:18.862347 22210 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4791316ded4254945739a555c6b533" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45007 } }
I20260925 13:11:18.862455 22210 leader_election.cc:302] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a4791316ded4254945739a555c6b533; no voters:
I20260925 13:11:18.862515 825 external_mini_cluster.cc:1428] Setting key b9b96b97b6cca89b8c10b56818298a67
I20260925 13:11:18.862623 22210 leader_election.cc:288] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:18.862687 22213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:18.862864 22213 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533 [term 1 LEADER]: Becoming Leader. State: Replica: 5a4791316ded4254945739a555c6b533, State: Running, Role: LEADER
I20260925 13:11:18.862917 22210 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:18.863054 22213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4791316ded4254945739a555c6b533" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45007 } }
I20260925 13:11:18.863557 22214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a4791316ded4254945739a555c6b533" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4791316ded4254945739a555c6b533" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45007 } } }
I20260925 13:11:18.863667 22214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:18.864019 22224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:18.864295 22215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a4791316ded4254945739a555c6b533. Latest consensus state: current_term: 1 leader_uuid: "5a4791316ded4254945739a555c6b533" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4791316ded4254945739a555c6b533" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45007 } } }
I20260925 13:11:18.864476 22215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:18.864528 22224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:18.865970 22224 catalog_manager.cc:1347] Generated new cluster ID: e626261a25f842cf833478ada0f47cd5
I20260925 13:11:18.866021 22224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:18.869539 22217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:18.863079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:55307 (local address 127.0.0.1:45007)
0925 13:11:18.863258 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:18.863268 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:18.864096 (+ 828us) server_negotiation.cc:407] Connection header received
0925 13:11:18.864232 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:18.864241 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:18.864317 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:18.864458 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:18.864925 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:18.866292 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:18.867384 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:18.868031 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:18.868145 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:18.868845 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:18.868849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:18.868849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:18.869053 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:18.869092 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:18.869106 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:18.869187 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:18.869309 (+ 122us) server_negotiation.cc:299] Negotiation successful
0925 13:11:18.869337 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260925 13:11:18.870448 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:18.874372 22224 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:18.875072 22224 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:18.877696 22217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:18.874192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:46237 (local address 127.0.0.1:45007)
0925 13:11:18.874226 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:18.874227 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:18.874400 (+ 173us) server_negotiation.cc:407] Connection header received
0925 13:11:18.874466 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:18.874467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:18.874477 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:18.874535 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:18.874663 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:18.875366 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:18.876371 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:18.876692 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:18.876747 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:18.877458 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:18.877459 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:18.877460 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:18.877464 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:18.877477 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:18.877481 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:18.877536 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:18.877618 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 13:11:18.877629 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:11:18.880667 22224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a4791316ded4254945739a555c6b533: Generated new TSK 0
I20260925 13:11:18.880836 22224 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:18.886168 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 22142
W20260925 13:11:18.889986 22127 connection.cc:535] client connection to 127.0.0.1:45007 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45007): Connection reset by peer (error 104)
2026-09-25T13:11:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1282 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:18 dist-test-slave-chkf krb5kdc[22241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:18 dist-test-slave-chkf krb5kdc[22241](info): set up 2 sockets
Sep 25 13:11:18 dist-test-slave-chkf krb5kdc[22241](info): commencing operation
krb5kdc: starting...
W20260925 13:11:19.910856 825 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:19 dist-test-slave-chkf krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 13:11:20.024928 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35979
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:20.105020 22254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:20.105221 22254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:20.105244 22254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:20.106386 22254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:20.106431 22254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:20.106444 22254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:20.106457 22254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:20.106469 22254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:20.108006 22254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35979
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:20.108260 22254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:20.108491 22254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:20.113420 22260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:20.113402 22259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:20.113628 22254 server_base.cc:1034] running on GCE node
W20260925 13:11:20.113373 22262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:20.113924 22254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:20.114203 22254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:20.115377 22254 hybrid_clock.cc:648] HybridClock initialized: now 1790341880115368 us; error 40 us; skew 500 ppm
Sep 25 13:11:20 dist-test-slave-chkf krb5kdc[22241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341880, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:20.117910 22254 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 13:11:20.118335 22254 webserver.cc:460] Webserver started at http://127.0.0.1:46695/ using document root <none> and password file <none>
I20260925 13:11:20.118608 22254 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:20.118680 22254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:20.118865 22254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:20.119944 22254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "bf7b636a872a4ae38f1191e66ba041c2"
format_stamp: "Formatted at 2026-09-25 13:11:20 on dist-test-slave-chkf"
server_key: "a2628fa0a3a0f2ccd82e39064b2b86e6"
server_key_iv: "ef7d1e043da4397da29bd8dfa48e3eef"
server_key_version: "encryptionkey@0"
I20260925 13:11:20.120293 22254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "bf7b636a872a4ae38f1191e66ba041c2"
format_stamp: "Formatted at 2026-09-25 13:11:20 on dist-test-slave-chkf"
server_key: "a2628fa0a3a0f2ccd82e39064b2b86e6"
server_key_iv: "ef7d1e043da4397da29bd8dfa48e3eef"
server_key_version: "encryptionkey@0"
I20260925 13:11:20.121610 22254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:11:20.122375 22269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:20.122627 22254 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:11:20.122778 22254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "bf7b636a872a4ae38f1191e66ba041c2"
format_stamp: "Formatted at 2026-09-25 13:11:20 on dist-test-slave-chkf"
server_key: "a2628fa0a3a0f2ccd82e39064b2b86e6"
server_key_iv: "ef7d1e043da4397da29bd8dfa48e3eef"
server_key_version: "encryptionkey@0"
I20260925 13:11:20.122869 22254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:20.126142 22254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:20.128177 22254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:20.128332 22254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:20.132697 22254 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35979
I20260925 13:11:20.132778 22321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35979 every 8 connection(s)
I20260925 13:11:20.133104 22254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:20.134115 22322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:20.136391 22322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2: Bootstrap starting.
I20260925 13:11:20.137038 22322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:20.137326 22322 log.cc:824] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:20.138058 22322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2: No bootstrap required, opened a new log
I20260925 13:11:20.139577 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 22254
I20260925 13:11:20.139670 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:20.139853 825 external_mini_cluster.cc:1428] Setting key 8848a58a898ad8e6f204132c6101accc
I20260925 13:11:20.139815 22322 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7b636a872a4ae38f1191e66ba041c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35979 } }
I20260925 13:11:20.140008 22322 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:20.140057 22322 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf7b636a872a4ae38f1191e66ba041c2, State: Initialized, Role: FOLLOWER
I20260925 13:11:20.140224 22322 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7b636a872a4ae38f1191e66ba041c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35979 } }
I20260925 13:11:20.140322 22322 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:20.140354 22322 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:20.140450 22322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:20.141407 22322 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7b636a872a4ae38f1191e66ba041c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35979 } }
I20260925 13:11:20.141526 22322 leader_election.cc:302] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf7b636a872a4ae38f1191e66ba041c2; no voters:
I20260925 13:11:20.141721 22322 leader_election.cc:288] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:20.141753 22327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:20.141937 22327 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2 [term 1 LEADER]: Becoming Leader. State: Replica: bf7b636a872a4ae38f1191e66ba041c2, State: Running, Role: LEADER
I20260925 13:11:20.142050 22322 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:20.142105 22326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:20.140340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:34421 (local address 127.0.0.1:35979)
0925 13:11:20.140522 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:20.140531 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:20.140604 (+ 73us) server_negotiation.cc:407] Connection header received
0925 13:11:20.140715 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:20.140721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:20.140755 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:20.140887 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:20.141621 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:20.141624 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:20.141625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:20.141629 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:20.141687 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:20.141699 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:20.141765 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:20.141835 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 13:11:20.141880 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260925 13:11:20.142057 22327 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7b636a872a4ae38f1191e66ba041c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35979 } }
I20260925 13:11:20.142521 22327 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf7b636a872a4ae38f1191e66ba041c2. Latest consensus state: current_term: 1 leader_uuid: "bf7b636a872a4ae38f1191e66ba041c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7b636a872a4ae38f1191e66ba041c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35979 } } }
I20260925 13:11:20.142592 22327 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:20.142531 22329 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf7b636a872a4ae38f1191e66ba041c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf7b636a872a4ae38f1191e66ba041c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35979 } } }
I20260925 13:11:20.142644 22329 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:20.142789 22335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:20.143229 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:20.143594 22335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:20.145076 22335 catalog_manager.cc:1347] Generated new cluster ID: b2259fa2d3b74631ba34a6e1093f014b
I20260925 13:11:20.145128 22335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:20.147867 22326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:20.144812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.193:59669 (local address 127.0.0.1:35979)
0925 13:11:20.144842 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:20.144844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:20.146911 (+ 2067us) server_negotiation.cc:407] Connection header received
0925 13:11:20.146975 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:20.146976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:20.146981 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:20.147039 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:20.147668 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:20.147669 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:20.147669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:20.147672 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:20.147684 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:20.147688 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:20.147739 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:20.147793 (+ 54us) server_negotiation.cc:299] Negotiation successful
0925 13:11:20.147806 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 13:11:20.161810 22335 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:20.161994 22335 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:20.167783 22335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf7b636a872a4ae38f1191e66ba041c2: Generated new TSK 0
I20260925 13:11:20.167979 22335 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:20.170832 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 22254
2026-09-25T13:11:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1285 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:20 dist-test-slave-chkf krb5kdc[22353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:20 dist-test-slave-chkf krb5kdc[22353](info): set up 2 sockets
Sep 25 13:11:20 dist-test-slave-chkf krb5kdc[22353](info): commencing operation
krb5kdc: starting...
W20260925 13:11:21.190800 825 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:21 dist-test-slave-chkf krb5kdc[22353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:11:21.301329 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:35033
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:35033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:21.379436 22366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:21.379617 22366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:21.379639 22366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:21.380692 22366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:21.380725 22366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:21.380739 22366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:21.380752 22366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:21.380764 22366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:21.382110 22366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:35033
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:35033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:21.382294 22366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:21.382457 22366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:21.387038 22374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:21.387022 22372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:21.387024 22371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:21.387194 22366 server_base.cc:1034] running on GCE node
I20260925 13:11:21.387559 22366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:21.387892 22366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:21.389061 22366 hybrid_clock.cc:648] HybridClock initialized: now 1790341881389048 us; error 25 us; skew 500 ppm
Sep 25 13:11:21 dist-test-slave-chkf krb5kdc[22353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341881, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:21.394199 22366 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:11:21.394609 22366 webserver.cc:460] Webserver started at http://127.0.206.126:34027/ using document root <none> and password file <none>
I20260925 13:11:21.394891 22366 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:21.394959 22366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:21.395119 22366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:21.396162 22366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "47a327b5ed5a404784739e48157c8819"
format_stamp: "Formatted at 2026-09-25 13:11:21 on dist-test-slave-chkf"
server_key: "a7f2a342eecfa4d8a0fd2aec43e6c6d5"
server_key_iv: "61a942168d0c0574efc46b36ab76fa31"
server_key_version: "encryptionkey@0"
I20260925 13:11:21.396514 22366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "47a327b5ed5a404784739e48157c8819"
format_stamp: "Formatted at 2026-09-25 13:11:21 on dist-test-slave-chkf"
server_key: "a7f2a342eecfa4d8a0fd2aec43e6c6d5"
server_key_iv: "61a942168d0c0574efc46b36ab76fa31"
server_key_version: "encryptionkey@0"
I20260925 13:11:21.397789 22366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 13:11:21.398450 22381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:21.398625 22366 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:11:21.398737 22366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "47a327b5ed5a404784739e48157c8819"
format_stamp: "Formatted at 2026-09-25 13:11:21 on dist-test-slave-chkf"
server_key: "a7f2a342eecfa4d8a0fd2aec43e6c6d5"
server_key_iv: "61a942168d0c0574efc46b36ab76fa31"
server_key_version: "encryptionkey@0"
I20260925 13:11:21.398838 22366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:21.412667 22366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:21.414531 22366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:21.414695 22366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:21.418399 22366 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:35033
I20260925 13:11:21.418464 22433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:35033 every 8 connection(s)
I20260925 13:11:21.418846 22366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:21.419646 22434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:21.421829 22434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819: Bootstrap starting.
I20260925 13:11:21.422396 22434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:21.422679 22434 log.cc:824] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:21.423364 22434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819: No bootstrap required, opened a new log
I20260925 13:11:21.424847 22434 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47a327b5ed5a404784739e48157c8819" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35033 } }
I20260925 13:11:21.424973 22434 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:21.424996 22434 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47a327b5ed5a404784739e48157c8819, State: Initialized, Role: FOLLOWER
I20260925 13:11:21.425125 22434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47a327b5ed5a404784739e48157c8819" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35033 } }
I20260925 13:11:21.425203 22434 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:21.425226 22434 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:21.425253 22434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:21.425608 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 22366
I20260925 13:11:21.425737 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:21.425779 22434 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47a327b5ed5a404784739e48157c8819" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35033 } }
I20260925 13:11:21.425912 22434 leader_election.cc:302] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47a327b5ed5a404784739e48157c8819; no voters:
I20260925 13:11:21.425956 825 external_mini_cluster.cc:1428] Setting key 8dd88968c4e58ef28ad700c669ccecff
I20260925 13:11:21.426087 22434 leader_election.cc:288] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:21.426111 22437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:21.426280 22437 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819 [term 1 LEADER]: Becoming Leader. State: Replica: 47a327b5ed5a404784739e48157c8819, State: Running, Role: LEADER
I20260925 13:11:21.426414 22434 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:21.426405 22437 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47a327b5ed5a404784739e48157c8819" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35033 } }
I20260925 13:11:21.427071 22439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47a327b5ed5a404784739e48157c8819. Latest consensus state: current_term: 1 leader_uuid: "47a327b5ed5a404784739e48157c8819" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47a327b5ed5a404784739e48157c8819" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35033 } } }
I20260925 13:11:21.427155 22439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:21.427198 22438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47a327b5ed5a404784739e48157c8819" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47a327b5ed5a404784739e48157c8819" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35033 } } }
I20260925 13:11:21.427266 22438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:21.427492 22448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:21.428049 22448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:21.429562 22448 catalog_manager.cc:1347] Generated new cluster ID: 4381a76a3c364c84889f6627e2ea85df
I20260925 13:11:21.429608 22448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 13:11:21 dist-test-slave-chkf krb5kdc[22353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:11:21.437373 22441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:21.426548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56847 (local address 127.0.206.126:35033)
0925 13:11:21.426794 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:21.426803 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:21.426834 (+ 31us) server_negotiation.cc:407] Connection header received
0925 13:11:21.426953 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:21.426960 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:21.427021 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:21.428078 (+ 1057us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:21.428176 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:21.429542 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:21.430276 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:21.430699 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:21.430837 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:21.433079 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:21.433082 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:21.433083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:21.433094 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:21.435298 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:21.435666 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:21.435669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:21.435669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:21.435767 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:21.435992 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:21.435993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:21.435993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:21.436308 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:21.436897 (+ 589us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:21.436985 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 13:11:21.437099 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":67,"threads_started":1}
I20260925 13:11:21.439301 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:21.448988 22441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:21.441514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58697 (local address 127.0.206.126:35033)
0925 13:11:21.441546 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:21.441548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:21.441690 (+ 142us) server_negotiation.cc:407] Connection header received
0925 13:11:21.441770 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:21.441771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:21.441779 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:21.441857 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:21.442513 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:21.443216 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:21.444198 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:21.444527 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:21.444575 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:21.446426 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:21.446428 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:21.446428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:21.446435 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:21.447870 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:21.448223 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:21.448224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:21.448224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:21.448275 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:21.448488 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:21.448489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:21.448489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:21.448592 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:21.448694 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:21.448816 (+ 122us) server_negotiation.cc:299] Negotiation successful
0925 13:11:21.448876 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260925 13:11:21.453886 22448 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:21.454547 22448 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:21.468113 22448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47a327b5ed5a404784739e48157c8819: Generated new TSK 0
I20260925 13:11:21.468312 22448 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:21.472764 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 22366
W20260925 13:11:21.476765 22351 connection.cc:535] client connection to 127.0.206.126:35033 recv error: Network error: failed to read from TLS socket (remote: 127.0.206.126:35033): Connection reset by peer (error 104)
2026-09-25T13:11:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1301 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:21 dist-test-slave-chkf krb5kdc[22465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:21 dist-test-slave-chkf krb5kdc[22465](info): set up 2 sockets
Sep 25 13:11:21 dist-test-slave-chkf krb5kdc[22465](info): commencing operation
krb5kdc: starting...
W20260925 13:11:22.502831 825 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:22 dist-test-slave-chkf krb5kdc[22465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:11:22.609143 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43633
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:43633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:22.686364 22478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:22.686596 22478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:22.686636 22478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:22.687808 22478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:22.687875 22478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:22.687902 22478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:22.687934 22478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:22.687958 22478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:22.689435 22478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:43633
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:22.689711 22478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:22.689918 22478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:22.694703 22486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:22.694748 22484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:22.694832 22478 server_base.cc:1034] running on GCE node
W20260925 13:11:22.694669 22483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:22.695165 22478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:22.695444 22478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:22.696621 22478 hybrid_clock.cc:648] HybridClock initialized: now 1790341882696606 us; error 26 us; skew 500 ppm
Sep 25 13:11:22 dist-test-slave-chkf krb5kdc[22465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341882, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:22.698951 22478 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:11:22.699389 22478 webserver.cc:460] Webserver started at http://127.0.206.126:41193/ using document root <none> and password file <none>
I20260925 13:11:22.699606 22478 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:22.699671 22478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:22.699820 22478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:22.700892 22478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "8d5e5541dc1146f9899690114a428e90"
format_stamp: "Formatted at 2026-09-25 13:11:22 on dist-test-slave-chkf"
server_key: "ae8fc27fe501d30b4f3cebff1019b41f"
server_key_iv: "29ac057727d2be26ead9832a5cc2116c"
server_key_version: "encryptionkey@0"
I20260925 13:11:22.701359 22478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "8d5e5541dc1146f9899690114a428e90"
format_stamp: "Formatted at 2026-09-25 13:11:22 on dist-test-slave-chkf"
server_key: "ae8fc27fe501d30b4f3cebff1019b41f"
server_key_iv: "29ac057727d2be26ead9832a5cc2116c"
server_key_version: "encryptionkey@0"
I20260925 13:11:22.702921 22478 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260925 13:11:22.703626 22493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:22.703789 22478 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:11:22.703891 22478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "8d5e5541dc1146f9899690114a428e90"
format_stamp: "Formatted at 2026-09-25 13:11:22 on dist-test-slave-chkf"
server_key: "ae8fc27fe501d30b4f3cebff1019b41f"
server_key_iv: "29ac057727d2be26ead9832a5cc2116c"
server_key_version: "encryptionkey@0"
I20260925 13:11:22.703961 22478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:22.723950 22478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:22.725652 22478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:22.725806 22478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:22.729796 22478 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:43633
I20260925 13:11:22.729826 22545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:43633 every 8 connection(s)
I20260925 13:11:22.730155 22478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:22.730993 22546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:22.733001 22546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90: Bootstrap starting.
I20260925 13:11:22.733120 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 22478
I20260925 13:11:22.733246 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:22.733408 825 external_mini_cluster.cc:1428] Setting key 84a5e855cf2bf9216516c1d53a339e35
I20260925 13:11:22.733652 22546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:22.733989 22546 log.cc:824] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:22.734644 22546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90: No bootstrap required, opened a new log
I20260925 13:11:22.736397 22546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d5e5541dc1146f9899690114a428e90" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43633 } }
I20260925 13:11:22.736542 22546 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:22.736593 22546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d5e5541dc1146f9899690114a428e90, State: Initialized, Role: FOLLOWER
I20260925 13:11:22.736709 22546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d5e5541dc1146f9899690114a428e90" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43633 } }
I20260925 13:11:22.736795 22546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:22.736816 22546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:22.736878 22546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:22.737495 22546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d5e5541dc1146f9899690114a428e90" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43633 } }
I20260925 13:11:22.737601 22546 leader_election.cc:302] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d5e5541dc1146f9899690114a428e90; no voters:
I20260925 13:11:22.737744 22546 leader_election.cc:288] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:22.737797 22551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:22.737939 22551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90 [term 1 LEADER]: Becoming Leader. State: Replica: 8d5e5541dc1146f9899690114a428e90, State: Running, Role: LEADER
I20260925 13:11:22.738020 22546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:22.738036 22551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d5e5541dc1146f9899690114a428e90" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43633 } }
I20260925 13:11:22.738420 22551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d5e5541dc1146f9899690114a428e90. Latest consensus state: current_term: 1 leader_uuid: "8d5e5541dc1146f9899690114a428e90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d5e5541dc1146f9899690114a428e90" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43633 } } }
I20260925 13:11:22.738468 22553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d5e5541dc1146f9899690114a428e90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d5e5541dc1146f9899690114a428e90" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43633 } } }
I20260925 13:11:22.738538 22551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:22.738646 22553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:22.739192 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:22.733897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37635 (local address 127.0.206.126:43633)
0925 13:11:22.734094 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:22.734103 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:22.734139 (+ 36us) server_negotiation.cc:407] Connection header received
0925 13:11:22.734671 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:22.734678 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:22.734786 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:22.734964 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:22.735155 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:22.736000 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:22.736929 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:22.737551 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:22.737671 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:22.738419 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:22.738422 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:22.738423 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:22.738634 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:22.738672 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:22.738687 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:22.738793 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:22.738986 (+ 193us) server_negotiation.cc:299] Negotiation successful
0925 13:11:22.739010 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":73,"threads_started":1}
I20260925 13:11:22.739470 22560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:22.740164 22560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:22.740296 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:22.741643 22560 catalog_manager.cc:1347] Generated new cluster ID: 593cec03602d4f0a924fa122d9ac6f66
I20260925 13:11:22.741690 22560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:22.745124 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:22.741804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47893 (local address 127.0.206.126:43633)
0925 13:11:22.741837 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:22.741838 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:22.741864 (+ 26us) server_negotiation.cc:407] Connection header received
0925 13:11:22.741934 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:22.741934 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:22.741942 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:22.741994 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:22.742119 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:22.742830 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:22.743839 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:22.744142 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:22.744202 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:22.744912 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:22.744913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:22.744914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:22.744917 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:22.744927 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:22.744930 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:22.744980 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:22.745048 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 13:11:22.745056 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:11:22.751217 22560 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:22.751880 22560 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:22.755033 22560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d5e5541dc1146f9899690114a428e90: Generated new TSK 0
I20260925 13:11:22.755208 22560 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:22.759282 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 22478
W20260925 13:11:22.763168 22463 connection.cc:535] client connection to 127.0.206.126:43633 recv error: Network error: failed to read from TLS socket (remote: 127.0.206.126:43633): Connection reset by peer (error 104)
2026-09-25T13:11:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1286 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:22 dist-test-slave-chkf krb5kdc[22577](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:22 dist-test-slave-chkf krb5kdc[22577](info): set up 2 sockets
Sep 25 13:11:22 dist-test-slave-chkf krb5kdc[22577](info): commencing operation
krb5kdc: starting...
W20260925 13:11:23.782557 825 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:23 dist-test-slave-chkf krb5kdc[22577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:11:23.894860 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38133
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:38133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:23.973268 22590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:23.973477 22590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:23.973515 22590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:23.974646 22590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:23.974709 22590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:23.974790 22590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:23.974834 22590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:23.974865 22590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:23.976279 22590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:38133
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:23.976536 22590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:23.976780 22590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:23.981463 22596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:23.981518 22595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:23.981564 22598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:23.981566 22590 server_base.cc:1034] running on GCE node
I20260925 13:11:23.981876 22590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:23.982090 22590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:23.983222 22590 hybrid_clock.cc:648] HybridClock initialized: now 1790341883983208 us; error 27 us; skew 500 ppm
Sep 25 13:11:23 dist-test-slave-chkf krb5kdc[22577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341883, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:23.985484 22590 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:11:23.985934 22590 webserver.cc:460] Webserver started at http://127.0.206.126:46003/ using document root <none> and password file <none>
I20260925 13:11:23.986138 22590 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:23.986181 22590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:23.986339 22590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:23.987370 22590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "7f532d66d83d49e2856300d937740dca"
format_stamp: "Formatted at 2026-09-25 13:11:23 on dist-test-slave-chkf"
server_key: "85b8dec65e92c593a440b31816ec7538"
server_key_iv: "e210067e3dab2da70d174dbdbcecdcc4"
server_key_version: "encryptionkey@0"
I20260925 13:11:23.987712 22590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "7f532d66d83d49e2856300d937740dca"
format_stamp: "Formatted at 2026-09-25 13:11:23 on dist-test-slave-chkf"
server_key: "85b8dec65e92c593a440b31816ec7538"
server_key_iv: "e210067e3dab2da70d174dbdbcecdcc4"
server_key_version: "encryptionkey@0"
I20260925 13:11:23.988932 22590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:11:23.989611 22605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:23.989780 22590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:11:23.989881 22590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "7f532d66d83d49e2856300d937740dca"
format_stamp: "Formatted at 2026-09-25 13:11:23 on dist-test-slave-chkf"
server_key: "85b8dec65e92c593a440b31816ec7538"
server_key_iv: "e210067e3dab2da70d174dbdbcecdcc4"
server_key_version: "encryptionkey@0"
I20260925 13:11:23.989960 22590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:23.993095 22590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:23.994799 22590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:23.994940 22590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:23.998746 22590 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:38133
I20260925 13:11:23.998805 22657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:38133 every 8 connection(s)
I20260925 13:11:23.999099 22590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:23.999974 22658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:24.001982 22658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca: Bootstrap starting.
I20260925 13:11:24.002507 22658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:24.002831 22658 log.cc:824] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:24.003449 22658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca: No bootstrap required, opened a new log
I20260925 13:11:24.004953 22658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f532d66d83d49e2856300d937740dca" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38133 } }
I20260925 13:11:24.005096 22658 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:24.005142 22658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f532d66d83d49e2856300d937740dca, State: Initialized, Role: FOLLOWER
I20260925 13:11:24.005298 22658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f532d66d83d49e2856300d937740dca" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38133 } }
I20260925 13:11:24.005391 22658 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:24.005435 22658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:24.005481 22658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:24.005961 22658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f532d66d83d49e2856300d937740dca" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38133 } }
I20260925 13:11:24.006070 22658 leader_election.cc:302] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f532d66d83d49e2856300d937740dca; no voters:
I20260925 13:11:24.006240 22658 leader_election.cc:288] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:24.006273 22661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:24.006430 22661 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca [term 1 LEADER]: Becoming Leader. State: Replica: 7f532d66d83d49e2856300d937740dca, State: Running, Role: LEADER
I20260925 13:11:24.006496 22658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:24.006578 22661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f532d66d83d49e2856300d937740dca" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38133 } }
I20260925 13:11:24.007117 22662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f532d66d83d49e2856300d937740dca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f532d66d83d49e2856300d937740dca" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38133 } } }
I20260925 13:11:24.007294 22663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f532d66d83d49e2856300d937740dca. Latest consensus state: current_term: 1 leader_uuid: "7f532d66d83d49e2856300d937740dca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f532d66d83d49e2856300d937740dca" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38133 } } }
I20260925 13:11:24.007458 22663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:24.007361 22662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:24.007691 22673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:24.008168 22673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:24.008677 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 22590
I20260925 13:11:24.008810 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:24.008980 825 external_mini_cluster.cc:1428] Setting key af92f4ec74b8efb98e6a99323cc65f12
I20260925 13:11:24.009541 22673 catalog_manager.cc:1347] Generated new cluster ID: 0e31667e6bf046b7b27f71adee28d226
I20260925 13:11:24.009596 22673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:24.012166 22681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:24.009370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50131 (local address 127.0.206.126:38133)
0925 13:11:24.010830 (+ 1460us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:24.010838 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:24.010872 (+ 34us) server_negotiation.cc:407] Connection header received
0925 13:11:24.010995 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:24.011002 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:24.011037 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:24.011175 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:24.011777 (+ 602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:24.011780 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:24.011780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:24.011784 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:24.011816 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:24.011830 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:24.011901 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:24.011962 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 13:11:24.012000 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1412,"thread_start_us":62,"threads_started":1}
I20260925 13:11:24.013016 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:24.015545 22681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:24.014412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51393 (local address 127.0.206.126:38133)
0925 13:11:24.014442 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:24.014444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:24.014556 (+ 112us) server_negotiation.cc:407] Connection header received
0925 13:11:24.014645 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:24.014646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:24.014659 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:24.014687 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:24.015323 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:24.015324 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:24.015325 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:24.015328 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:24.015340 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:24.015344 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:24.015373 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:24.015476 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 13:11:24.015489 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260925 13:11:24.024066 22673 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:24.024214 22673 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:24.028363 22673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f532d66d83d49e2856300d937740dca: Generated new TSK 0
I20260925 13:11:24.028532 22673 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:24.031586 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 22590
2026-09-25T13:11:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1272 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:24 dist-test-slave-chkf krb5kdc[22689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:24 dist-test-slave-chkf krb5kdc[22689](info): set up 2 sockets
Sep 25 13:11:24 dist-test-slave-chkf krb5kdc[22689](info): commencing operation
krb5kdc: starting...
W20260925 13:11:25.046855 825 mini_kdc.cc:121] Time spent starting KDC: real 0.998s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:25 dist-test-slave-chkf krb5kdc[22689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:11:25.161499 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:37277
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:37277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:25.240814 22702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:25.241016 22702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:25.241039 22702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:25.242101 22702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:25.242134 22702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:25.242148 22702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:25.242161 22702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:25.242172 22702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:25.243549 22702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:37277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:37277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:25.243741 22702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:25.243894 22702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:25.248596 22710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:25.248607 22707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:25.248697 22708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:25.248740 22702 server_base.cc:1034] running on GCE node
I20260925 13:11:25.249083 22702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:25.249409 22702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:25.250564 22702 hybrid_clock.cc:648] HybridClock initialized: now 1790341885250553 us; error 28 us; skew 500 ppm
Sep 25 13:11:25 dist-test-slave-chkf krb5kdc[22689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341885, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:25.253068 22702 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:11:25.253521 22702 webserver.cc:460] Webserver started at http://127.0.206.126:43563/ using document root <none> and password file <none>
I20260925 13:11:25.253751 22702 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:25.253818 22702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:25.253948 22702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:25.255007 22702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "86a4ba1e3b8d4e0f87456e67ad7996d2"
format_stamp: "Formatted at 2026-09-25 13:11:25 on dist-test-slave-chkf"
server_key: "8326d65ce24697db64fb737ca208ea25"
server_key_iv: "f7b42bf143930eb1cf7daf89e408d280"
server_key_version: "encryptionkey@0"
I20260925 13:11:25.255374 22702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "86a4ba1e3b8d4e0f87456e67ad7996d2"
format_stamp: "Formatted at 2026-09-25 13:11:25 on dist-test-slave-chkf"
server_key: "8326d65ce24697db64fb737ca208ea25"
server_key_iv: "f7b42bf143930eb1cf7daf89e408d280"
server_key_version: "encryptionkey@0"
I20260925 13:11:25.256670 22702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:11:25.257359 22717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:25.257542 22702 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:11:25.257639 22702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "86a4ba1e3b8d4e0f87456e67ad7996d2"
format_stamp: "Formatted at 2026-09-25 13:11:25 on dist-test-slave-chkf"
server_key: "8326d65ce24697db64fb737ca208ea25"
server_key_iv: "f7b42bf143930eb1cf7daf89e408d280"
server_key_version: "encryptionkey@0"
I20260925 13:11:25.257720 22702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:25.284062 22702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:25.285848 22702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:25.286010 22702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:25.289944 22702 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:37277
I20260925 13:11:25.289973 22769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:37277 every 8 connection(s)
I20260925 13:11:25.290306 22702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:25.291162 22770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:25.293262 22770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2: Bootstrap starting.
I20260925 13:11:25.293809 22770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:25.294086 22770 log.cc:824] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:25.294777 22770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2: No bootstrap required, opened a new log
I20260925 13:11:25.295991 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 22702
I20260925 13:11:25.296113 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:25.296305 825 external_mini_cluster.cc:1428] Setting key a90cfc76c86cbdf14ed159568822c00f
I20260925 13:11:25.296639 22770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a4ba1e3b8d4e0f87456e67ad7996d2" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37277 } }
I20260925 13:11:25.296840 22770 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:25.296885 22770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86a4ba1e3b8d4e0f87456e67ad7996d2, State: Initialized, Role: FOLLOWER
I20260925 13:11:25.297050 22770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a4ba1e3b8d4e0f87456e67ad7996d2" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37277 } }
I20260925 13:11:25.297113 22770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:25.297158 22770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:25.297204 22770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:25.297910 22770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a4ba1e3b8d4e0f87456e67ad7996d2" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37277 } }
I20260925 13:11:25.298019 22770 leader_election.cc:302] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86a4ba1e3b8d4e0f87456e67ad7996d2; no voters:
I20260925 13:11:25.298166 22770 leader_election.cc:288] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:25.298203 22775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:25.298385 22775 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2 [term 1 LEADER]: Becoming Leader. State: Replica: 86a4ba1e3b8d4e0f87456e67ad7996d2, State: Running, Role: LEADER
I20260925 13:11:25.298403 22770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:25.299068 22775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a4ba1e3b8d4e0f87456e67ad7996d2" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37277 } }
I20260925 13:11:25.299639 22777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86a4ba1e3b8d4e0f87456e67ad7996d2. Latest consensus state: current_term: 1 leader_uuid: "86a4ba1e3b8d4e0f87456e67ad7996d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a4ba1e3b8d4e0f87456e67ad7996d2" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37277 } } }
I20260925 13:11:25.299639 22776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86a4ba1e3b8d4e0f87456e67ad7996d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a4ba1e3b8d4e0f87456e67ad7996d2" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37277 } } }
I20260925 13:11:25.299878 22776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:25.299875 22777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2 [sys.catalog]: This master's current role is: LEADER
W20260925 13:11:25.300176 22789 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:11:25.300227 22789 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:11:25.300316 22790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:25.300539 22790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:25.302826 22790 catalog_manager.cc:1347] Generated new cluster ID: cbdb1b1dccd7482f9d6b45d9298b8950
I20260925 13:11:25.302886 22790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 13:11:25 dist-test-slave-chkf krb5kdc[22689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:11:25.308043 22774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:25.296789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46003 (local address 127.0.206.126:37277)
0925 13:11:25.296995 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:25.297004 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:25.297059 (+ 55us) server_negotiation.cc:407] Connection header received
0925 13:11:25.297197 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:25.297204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:25.297258 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:25.297379 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:25.298164 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:25.299865 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:25.300950 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:25.301552 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:25.301664 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:25.304269 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:25.304273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:25.304273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:25.304284 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:25.306385 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:25.306811 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:25.306814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:25.306814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:25.306932 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:25.307141 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:25.307142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:25.307143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:25.307461 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:25.307634 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:25.307709 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 13:11:25.307855 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":112,"threads_started":1}
I20260925 13:11:25.308928 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:25.310626 22790 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:25.311146 22790 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:25.313511 22790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86a4ba1e3b8d4e0f87456e67ad7996d2: Generated new TSK 0
I20260925 13:11:25.313700 22790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:25.321494 22774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:25.315006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38757 (local address 127.0.206.126:37277)
0925 13:11:25.315050 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:25.315052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:25.315177 (+ 125us) server_negotiation.cc:407] Connection header received
0925 13:11:25.315228 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:25.315228 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:25.315250 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:25.315297 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:25.315863 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:25.316299 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:25.317328 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:25.317513 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:25.317559 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:25.318829 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:25.318830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:25.318830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:25.318837 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:25.320444 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:25.320845 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:25.320846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:25.320846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:25.320897 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:25.321096 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:25.321097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:25.321097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:25.321186 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:25.321269 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:25.321384 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 13:11:25.321442 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 13:11:25.323536 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 22702
2026-09-25T13:11:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1291 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:25 dist-test-slave-chkf krb5kdc[22801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:25 dist-test-slave-chkf krb5kdc[22801](info): set up 2 sockets
Sep 25 13:11:25 dist-test-slave-chkf krb5kdc[22801](info): commencing operation
krb5kdc: starting...
W20260925 13:11:26.342901 825 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:26 dist-test-slave-chkf krb5kdc[22801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:11:26.459545 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:39955
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:39955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:26.539106 22814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:26.539319 22814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:26.539342 22814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:26.540552 22814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:26.540593 22814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:26.540607 22814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:26.540620 22814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:26.540632 22814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:26.542153 22814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:39955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:39955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:26.542407 22814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:26.542632 22814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:26.547391 22820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:26.547430 22822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:26.547442 22819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:26.547593 22814 server_base.cc:1034] running on GCE node
I20260925 13:11:26.547844 22814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:26.548141 22814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:26.549321 22814 hybrid_clock.cc:648] HybridClock initialized: now 1790341886549300 us; error 38 us; skew 500 ppm
Sep 25 13:11:26 dist-test-slave-chkf krb5kdc[22801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341886, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:26.559168 22814 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:11:26.559695 22814 webserver.cc:460] Webserver started at http://127.0.206.126:37385/ using document root <none> and password file <none>
I20260925 13:11:26.559933 22814 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:26.560005 22814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:26.560163 22814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:26.561309 22814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "eed516724d29404f8a49e909b28f6aed"
format_stamp: "Formatted at 2026-09-25 13:11:26 on dist-test-slave-chkf"
server_key: "7ef38d51c84e87f6f244b881eb429c35"
server_key_iv: "00f4bd2717938d021f2e0eca6a8f24cd"
server_key_version: "encryptionkey@0"
I20260925 13:11:26.561688 22814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "eed516724d29404f8a49e909b28f6aed"
format_stamp: "Formatted at 2026-09-25 13:11:26 on dist-test-slave-chkf"
server_key: "7ef38d51c84e87f6f244b881eb429c35"
server_key_iv: "00f4bd2717938d021f2e0eca6a8f24cd"
server_key_version: "encryptionkey@0"
I20260925 13:11:26.563040 22814 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:11:26.563861 22829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:26.564044 22814 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:11:26.564146 22814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "eed516724d29404f8a49e909b28f6aed"
format_stamp: "Formatted at 2026-09-25 13:11:26 on dist-test-slave-chkf"
server_key: "7ef38d51c84e87f6f244b881eb429c35"
server_key_iv: "00f4bd2717938d021f2e0eca6a8f24cd"
server_key_version: "encryptionkey@0"
I20260925 13:11:26.564229 22814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:26.593528 22814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:26.595672 22814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:26.595837 22814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:26.599773 22814 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:39955
I20260925 13:11:26.599813 22881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:39955 every 8 connection(s)
I20260925 13:11:26.600191 22814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:26.601022 22882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:26.603236 22882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed: Bootstrap starting.
I20260925 13:11:26.603739 22882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:26.603971 22882 log.cc:824] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:26.604359 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 22814
I20260925 13:11:26.604503 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:26.604696 825 external_mini_cluster.cc:1428] Setting key 54d9a77be264addcd86e92abc168b61f
I20260925 13:11:26.604756 22882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed: No bootstrap required, opened a new log
I20260925 13:11:26.607292 22882 raft_consensus.cc:348] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eed516724d29404f8a49e909b28f6aed" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39955 } }
I20260925 13:11:26.607448 22882 raft_consensus.cc:374] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:26.607487 22882 raft_consensus.cc:729] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eed516724d29404f8a49e909b28f6aed, State: Initialized, Role: FOLLOWER
I20260925 13:11:26.607637 22882 consensus_queue.cc:260] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eed516724d29404f8a49e909b28f6aed" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39955 } }
I20260925 13:11:26.607717 22882 raft_consensus.cc:388] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:26.607748 22882 raft_consensus.cc:482] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:26.607808 22882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:26.608450 22882 raft_consensus.cc:504] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eed516724d29404f8a49e909b28f6aed" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39955 } }
I20260925 13:11:26.608562 22882 leader_election.cc:302] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eed516724d29404f8a49e909b28f6aed; no voters:
I20260925 13:11:26.608760 22882 leader_election.cc:288] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:26.608784 22887 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:26.608901 22887 raft_consensus.cc:686] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed [term 1 LEADER]: Becoming Leader. State: Replica: eed516724d29404f8a49e909b28f6aed, State: Running, Role: LEADER
I20260925 13:11:26.609038 22882 sys_catalog.cc:564] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:26.609035 22887 consensus_queue.cc:237] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eed516724d29404f8a49e909b28f6aed" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39955 } }
I20260925 13:11:26.609524 22888 sys_catalog.cc:455] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed [sys.catalog]: SysCatalogTable state changed. Reason: New leader eed516724d29404f8a49e909b28f6aed. Latest consensus state: current_term: 1 leader_uuid: "eed516724d29404f8a49e909b28f6aed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eed516724d29404f8a49e909b28f6aed" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39955 } } }
I20260925 13:11:26.609596 22888 sys_catalog.cc:458] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:26.609807 22891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:26.609792 22889 sys_catalog.cc:455] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eed516724d29404f8a49e909b28f6aed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eed516724d29404f8a49e909b28f6aed" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39955 } } }
I20260925 13:11:26.610239 22889 sys_catalog.cc:458] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:26.610585 22891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:26.610845 22886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:26.605191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36073 (local address 127.0.206.126:39955)
0925 13:11:26.605347 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:26.605356 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:26.605393 (+ 37us) server_negotiation.cc:407] Connection header received
0925 13:11:26.606011 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:26.606018 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:26.606076 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:26.606191 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:26.606444 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:26.607355 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:26.608503 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:26.609153 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:26.609564 (+ 411us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:26.609999 (+ 435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:26.610003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:26.610003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:26.610220 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:26.610262 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:26.610277 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:26.610520 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:26.610557 (+ 37us) server_negotiation.cc:299] Negotiation successful
0925 13:11:26.610625 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":83,"threads_started":1}
I20260925 13:11:26.611939 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:26.612236 22891 catalog_manager.cc:1347] Generated new cluster ID: c14d43edb9574b7ca70533775759faa8
I20260925 13:11:26.612294 22891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:26.616187 22886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:26.613364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52599 (local address 127.0.206.126:39955)
0925 13:11:26.613399 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:26.613401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:26.613515 (+ 114us) server_negotiation.cc:407] Connection header received
0925 13:11:26.613592 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:26.613593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:26.613604 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:26.613659 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:26.613798 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:26.614298 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:26.615074 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:26.615276 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:26.615347 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:26.615940 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:26.615941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:26.615941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:26.615946 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:26.615958 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:26.615962 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:26.616004 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:26.616064 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 13:11:26.616093 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:11:26.624756 22891 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:26.625452 22891 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:26.629341 22891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eed516724d29404f8a49e909b28f6aed: Generated new TSK 0
I20260925 13:11:26.629506 22891 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:26.633639 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 22814
2026-09-25T13:11:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:26 dist-test-slave-chkf krb5kdc[22913](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:26 dist-test-slave-chkf krb5kdc[22913](info): set up 2 sockets
Sep 25 13:11:26 dist-test-slave-chkf krb5kdc[22913](info): commencing operation
krb5kdc: starting...
W20260925 13:11:27.666836 825 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:27 dist-test-slave-chkf krb5kdc[22913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:11:27.775453 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43749
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:32955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:43749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:27.852727 22926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:27.852911 22926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:27.852933 22926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:27.854075 22926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:27.854120 22926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:27.854133 22926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:27.854146 22926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:27.854156 22926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:27.855683 22926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:32955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:43749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:27.855916 22926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:27.856074 22926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:27.860589 22931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:27.860601 22934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:27.860688 22932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:27.860756 22926 server_base.cc:1034] running on GCE node
I20260925 13:11:27.861018 22926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:27.861295 22926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:27.862462 22926 hybrid_clock.cc:648] HybridClock initialized: now 1790341887862445 us; error 30 us; skew 500 ppm
Sep 25 13:11:27 dist-test-slave-chkf krb5kdc[22913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341887, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:27.864900 22926 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:11:27.865371 22926 webserver.cc:460] Webserver started at http://127.0.206.126:42229/ using document root <none> and password file <none>
I20260925 13:11:27.865589 22926 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:27.865656 22926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:27.865818 22926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:27.866896 22926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "8ccf37f4e3834af68e03af41a6b90e9f"
format_stamp: "Formatted at 2026-09-25 13:11:27 on dist-test-slave-chkf"
server_key: "c00800c4407fabab5b9433dae3ad71e7"
server_key_iv: "386793bd0ed41c6121eadba5ce9babf4"
server_key_version: "encryptionkey@0"
I20260925 13:11:27.867244 22926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "8ccf37f4e3834af68e03af41a6b90e9f"
format_stamp: "Formatted at 2026-09-25 13:11:27 on dist-test-slave-chkf"
server_key: "c00800c4407fabab5b9433dae3ad71e7"
server_key_iv: "386793bd0ed41c6121eadba5ce9babf4"
server_key_version: "encryptionkey@0"
I20260925 13:11:27.868561 22926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 13:11:27.869302 22941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:27.869472 22926 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:11:27.869576 22926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "8ccf37f4e3834af68e03af41a6b90e9f"
format_stamp: "Formatted at 2026-09-25 13:11:27 on dist-test-slave-chkf"
server_key: "c00800c4407fabab5b9433dae3ad71e7"
server_key_iv: "386793bd0ed41c6121eadba5ce9babf4"
server_key_version: "encryptionkey@0"
I20260925 13:11:27.869649 22926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:27.872896 22926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:27.874644 22926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:27.874822 22926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:27.878649 22926 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:43749
I20260925 13:11:27.878679 22993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:43749 every 8 connection(s)
I20260925 13:11:27.879030 22926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:27.879951 22994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:27.882045 22994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f: Bootstrap starting.
I20260925 13:11:27.882602 22994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:27.882936 22994 log.cc:824] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:27.883576 22994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f: No bootstrap required, opened a new log
I20260925 13:11:27.885046 22994 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ccf37f4e3834af68e03af41a6b90e9f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43749 } }
I20260925 13:11:27.885174 22994 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 13:11:27.885196 22994 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ccf37f4e3834af68e03af41a6b90e9f, State: Initialized, Role: FOLLOWER
I20260925 13:11:27.885288 22994 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ccf37f4e3834af68e03af41a6b90e9f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43749 } }
I20260925 13:11:27.885351 22994 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 13:11:27.885387 22994 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 13:11:27.885423 22994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:27.885917 22994 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ccf37f4e3834af68e03af41a6b90e9f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43749 } }
I20260925 13:11:27.886019 22994 leader_election.cc:302] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ccf37f4e3834af68e03af41a6b90e9f; no voters:
I20260925 13:11:27.886186 22994 leader_election.cc:288] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 13:11:27.886229 22997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:27.886435 22997 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f [term 1 LEADER]: Becoming Leader. State: Replica: 8ccf37f4e3834af68e03af41a6b90e9f, State: Running, Role: LEADER
I20260925 13:11:27.886420 22994 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:27.886595 22997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ccf37f4e3834af68e03af41a6b90e9f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43749 } }
I20260925 13:11:27.887013 22999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ccf37f4e3834af68e03af41a6b90e9f. Latest consensus state: current_term: 1 leader_uuid: "8ccf37f4e3834af68e03af41a6b90e9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ccf37f4e3834af68e03af41a6b90e9f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43749 } } }
I20260925 13:11:27.887137 22999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:27.887162 22998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ccf37f4e3834af68e03af41a6b90e9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ccf37f4e3834af68e03af41a6b90e9f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43749 } } }
I20260925 13:11:27.887270 22998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:27.887601 23006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:27.888069 23006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:27.889127 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 22926
I20260925 13:11:27.889268 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:27.889460 825 external_mini_cluster.cc:1428] Setting key ea222aee6a55818171be19f0c9875bcd
I20260925 13:11:27.889535 23006 catalog_manager.cc:1347] Generated new cluster ID: 5ce34adc09364d5e913b94ca0405e754
I20260925 13:11:27.889571 23006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:27.896255 23017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:27.889891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36515 (local address 127.0.206.126:43749)
0925 13:11:27.894852 (+ 4961us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:27.894861 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:27.894895 (+ 34us) server_negotiation.cc:407] Connection header received
0925 13:11:27.895003 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:27.895009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:27.895037 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:27.895187 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:27.895829 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:27.895833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:27.895833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:27.895837 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:27.895877 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:27.895891 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:27.895958 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:27.896032 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 13:11:27.896068 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4911,"thread_start_us":57,"threads_started":1}
I20260925 13:11:27.897096 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:27.899452 23017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:27.898463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37849 (local address 127.0.206.126:43749)
0925 13:11:27.898494 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:27.898496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:27.898601 (+ 105us) server_negotiation.cc:407] Connection header received
0925 13:11:27.898653 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:27.898653 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:27.898657 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:27.898736 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:27.899271 (+ 535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:27.899272 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:27.899273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 13:11:27.899276 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:27.899288 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 13:11:27.899291 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:27.899341 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:27.899391 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 13:11:27.899400 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 13:11:27.903627 23006 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:27.903764 23006 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:27.910187 23006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ccf37f4e3834af68e03af41a6b90e9f: Generated new TSK 0
I20260925 13:11:27.910352 23006 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:27.913120 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 22926
2026-09-25T13:11:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1279 ms)
[----------] 18 tests from AuthTokenIssuingTest (23297 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:27 dist-test-slave-chkf krb5kdc[23025](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:27 dist-test-slave-chkf krb5kdc[23025](info): set up 2 sockets
Sep 25 13:11:27 dist-test-slave-chkf krb5kdc[23025](info): commencing operation
krb5kdc: starting...
W20260925 13:11:28.930876 825 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:29 dist-test-slave-chkf krb5kdc[23025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:11:29.043232 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38555
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:41911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:38555,127.0.206.125:41079,127.0.206.124:38457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:29.119549 23038 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 13:11:29.119799 23038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:29.119834 23038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:29.119874 23038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:29.121106 23038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:29.121178 23038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:29.121201 23038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:29.121222 23038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:29.121249 23038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:29.122682 23038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:38555,127.0.206.125:41079,127.0.206.124:38457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:29.122987 23038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:29.123217 23038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:29.127980 23044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:29.127991 23043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:29.127992 23046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:29.128180 23038 server_base.cc:1034] running on GCE node
I20260925 13:11:29.128430 23038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:29.128714 23038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:29.129855 23038 hybrid_clock.cc:648] HybridClock initialized: now 1790341889129845 us; error 27 us; skew 500 ppm
Sep 25 13:11:29 dist-test-slave-chkf krb5kdc[23025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:29.132294 23038 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:11:29.132710 23038 webserver.cc:460] Webserver started at http://127.0.206.126:39511/ using document root <none> and password file <none>
I20260925 13:11:29.132936 23038 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:29.132979 23038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:29.133113 23038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:29.134176 23038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "7a4531c5265f4ab499e97ef13675ebb4"
format_stamp: "Formatted at 2026-09-25 13:11:29 on dist-test-slave-chkf"
server_key: "cc12d4e44267ce9cc568cc393d5f30f0"
server_key_iv: "b5f914171ba2340193f9b49d583c2ac9"
server_key_version: "encryptionkey@0"
I20260925 13:11:29.134529 23038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "7a4531c5265f4ab499e97ef13675ebb4"
format_stamp: "Formatted at 2026-09-25 13:11:29 on dist-test-slave-chkf"
server_key: "cc12d4e44267ce9cc568cc393d5f30f0"
server_key_iv: "b5f914171ba2340193f9b49d583c2ac9"
server_key_version: "encryptionkey@0"
I20260925 13:11:29.135874 23038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:11:29.136611 23053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:29.136791 23038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:11:29.136875 23038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "7a4531c5265f4ab499e97ef13675ebb4"
format_stamp: "Formatted at 2026-09-25 13:11:29 on dist-test-slave-chkf"
server_key: "cc12d4e44267ce9cc568cc393d5f30f0"
server_key_iv: "b5f914171ba2340193f9b49d583c2ac9"
server_key_version: "encryptionkey@0"
I20260925 13:11:29.136929 23038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:29.178493 23038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:29.180271 23038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:29.180377 23038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:29.184054 23038 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:38555
I20260925 13:11:29.184115 23105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:38555 every 8 connection(s)
I20260925 13:11:29.184422 23038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:29.185390 23106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:29.186895 23106 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:11:29.187326 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 23038
I20260925 13:11:29.187441 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:29.187656 825 external_mini_cluster.cc:1428] Setting key e638fece684de4b6ef42e61317751ada
Sep 25 13:11:29 dist-test-slave-chkf krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.206.125@KRBTEST.COM; defaulting to no policy
I20260925 13:11:29.197445 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.187597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38555 (local address 127.0.0.1:49616)
0925 13:11:29.187833 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.187881 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:29.188636 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:29.188920 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:29.188923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:29.189283 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:29.189478 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:29.189483 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.190813 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.190814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:29.191481 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:29.191483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.192183 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.192184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:29.192234 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.192737 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:29.192744 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:29.194012 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:29.196070 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:29.196076 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:29.196077 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:29.196267 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:29.196503 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:29.196504 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:29.196504 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:29.196598 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:29.196987 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:29.196993 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:29.197049 (+ 56us) client_negotiation.cc:769] Sending connection context
0925 13:11:29.197127 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 13:11:29.197242 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":77,"threads_started":1}
I20260925 13:11:29.197448 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.187597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49616 (local address 127.0.206.126:38555)
0925 13:11:29.187784 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.187794 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.187984 (+ 190us) server_negotiation.cc:407] Connection header received
0925 13:11:29.188782 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.188784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.188800 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.188874 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.189589 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.190690 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.191561 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.192069 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.192194 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.194115 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:29.194122 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:29.194122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:29.194130 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:29.195990 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.196342 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.196345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.196345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.196419 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.196701 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.196702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.196703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.196907 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:29.196971 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:29.197173 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 13:11:29.197299 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.125@KRBTEST.COM" created.
I20260925 13:11:29.198334 23106 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:11:29.198640 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.198529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:41079 (local address 127.0.0.1:42982)
0925 13:11:29.198575 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.198615 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 13:11:29.198786 23055 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41079: connect: Connection refused (error 111)
I20260925 13:11:29.199187 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.199093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:41079 (local address 127.0.0.1:42984)
0925 13:11:29.199144 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.199168 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 13:11:29.199295 23106 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.125:41079: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41079: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.125.keytab.
Entry for principal kudu/127.0.206.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.125.keytab.
I20260925 13:11:29.211038 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.125:41079
--webserver_interface=127.0.206.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.125.keytab
--principal=kudu/127.0.206.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:41911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:38555,127.0.206.125:41079,127.0.206.124:38457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:11:29.248497 23106 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } attempt: 1
I20260925 13:11:29.248953 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.248822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:41079 (local address 127.0.0.1:42992)
0925 13:11:29.248890 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.248927 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 13:11:29.249342 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.249207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:41079 (local address 127.0.0.1:43006)
0925 13:11:29.249268 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.249310 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 13:11:29.249480 23106 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.125:41079: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41079: connect: Connection refused (error 111)
W20260925 13:11:29.301492 23113 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 13:11:29.301753 23113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:29.301777 23113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:29.301803 23113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:29.303395 23113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:29.303475 23113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:29.303505 23113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:29.303524 23113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:29.303558 23113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:29.305510 23113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:38555,127.0.206.125:41079,127.0.206.124:38457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.125.keytab
--principal=kudu/127.0.206.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.125:41079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.206.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:29.305758 23113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:29.306003 23113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:29.311244 23121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:29.311239 23119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:29.311239 23118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:29.311646 23113 server_base.cc:1034] running on GCE node
I20260925 13:11:29.311920 23113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:29.312259 23113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:29.313438 23113 hybrid_clock.cc:648] HybridClock initialized: now 1790341889313410 us; error 43 us; skew 500 ppm
Sep 25 13:11:29 dist-test-slave-chkf krb5kdc[23025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:29.316491 23113 init.cc:377] Logged in from keytab as kudu/127.0.206.125@KRBTEST.COM (short username kudu)
I20260925 13:11:29.316948 23113 webserver.cc:460] Webserver started at http://127.0.206.125:37445/ using document root <none> and password file <none>
I20260925 13:11:29.317178 23113 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:29.317245 23113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:29.317375 23113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:29.317642 23106 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } attempt: 2
I20260925 13:11:29.318015 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.317873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:41079 (local address 127.0.0.1:43014)
0925 13:11:29.317942 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.317979 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 13:11:29.318361 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.318255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:41079 (local address 127.0.0.1:43018)
0925 13:11:29.318306 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.318337 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 13:11:29.318455 23106 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.125:41079: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41079: connect: Connection refused (error 111)
I20260925 13:11:29.318620 23113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-1/data/instance:
uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe"
format_stamp: "Formatted at 2026-09-25 13:11:29 on dist-test-slave-chkf"
server_key: "1cc85efea83ddf905731de507b6781e7"
server_key_iv: "63ec2dbc3f2c607a560164995fcad66f"
server_key_version: "encryptionkey@0"
I20260925 13:11:29.318982 23113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-1/wal/instance:
uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe"
format_stamp: "Formatted at 2026-09-25 13:11:29 on dist-test-slave-chkf"
server_key: "1cc85efea83ddf905731de507b6781e7"
server_key_iv: "63ec2dbc3f2c607a560164995fcad66f"
server_key_version: "encryptionkey@0"
I20260925 13:11:29.320158 23113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:11:29.320859 23128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:29.321022 23113 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:11:29.321107 23113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-1/wal
uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe"
format_stamp: "Formatted at 2026-09-25 13:11:29 on dist-test-slave-chkf"
server_key: "1cc85efea83ddf905731de507b6781e7"
server_key_iv: "63ec2dbc3f2c607a560164995fcad66f"
server_key_version: "encryptionkey@0"
I20260925 13:11:29.321158 23113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:29.332870 23113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:29.334988 23113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:29.335090 23113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:29.338801 23113 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.125:41079
I20260925 13:11:29.338857 23180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.125:41079 every 8 connection(s)
I20260925 13:11:29.339148 23113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-1/data/info.pb
I20260925 13:11:29.340142 23181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:29.341603 23181 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:11:29.345428 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 23113
I20260925 13:11:29.345584 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-1/wal/instance
I20260925 13:11:29.345796 825 external_mini_cluster.cc:1428] Setting key 36e274d48217f5ba7d1bf47a514dabcd
Sep 25 13:11:29 dist-test-slave-chkf krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:11:29.350318 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.342164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38555 (local address 127.0.0.1:49626)
0925 13:11:29.342335 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.342368 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:29.342861 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:29.343065 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:29.343068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:29.343286 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:29.343461 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:29.343464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.344094 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.344095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:29.344849 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:29.344851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.345192 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.345193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:29.345317 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.345909 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:29.345920 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:29.347244 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:29.348930 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:29.348938 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:29.348940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:29.349192 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:29.349453 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:29.349454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:29.349454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:29.349553 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:29.349867 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:29.349871 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:29.349974 (+ 103us) client_negotiation.cc:769] Sending connection context
0925 13:11:29.350055 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 13:11:29.350163 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260925 13:11:29.350361 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.342200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49626 (local address 127.0.206.126:38555)
0925 13:11:29.342274 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.342277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.342444 (+ 167us) server_negotiation.cc:407] Connection header received
0925 13:11:29.342942 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.342943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.342955 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.343007 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.343528 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.344018 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.344918 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.345110 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.345161 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.347355 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:29.347357 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:29.347357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:29.347367 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:29.348806 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.349293 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.349295 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.349296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.349373 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.349624 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.349625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.349625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.349796 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:29.349875 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:29.350102 (+ 227us) server_negotiation.cc:299] Negotiation successful
0925 13:11:29.350211 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 13:11:29.351058 23181 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.206.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.124@KRBTEST.COM" created.
Sep 25 13:11:29 dist-test-slave-chkf krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
I20260925 13:11:29.359993 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.351352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:41079 (local address 127.0.0.1:43034)
0925 13:11:29.351407 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.351417 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:29.351460 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:29.351702 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:29.351703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:29.351841 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:29.351880 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:29.351881 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.352711 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.352712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:29.353199 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:29.353200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.353674 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.353675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:29.353738 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.354136 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:29.354141 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:29.356836 (+ 2695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:29.358987 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:29.358988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:29.358989 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:29.359141 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:29.359359 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:29.359360 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:29.359360 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:29.359407 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:29.359742 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:29.359743 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:29.359793 (+ 50us) client_negotiation.cc:769] Sending connection context
0925 13:11:29.359870 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 13:11:29.359935 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 13:11:29.360234 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.351419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43034 (local address 127.0.206.125:41079)
0925 13:11:29.351576 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.351580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.351594 (+ 14us) server_negotiation.cc:407] Connection header received
0925 13:11:29.351609 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.351611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.351621 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.351701 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.351957 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.352607 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.353283 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.353587 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.353675 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.356938 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:29.356946 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:29.356946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:29.356951 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:29.358912 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.359228 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.359231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.359231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.359293 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.359469 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.359470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.359471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.359648 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:29.359737 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:29.359922 (+ 185us) server_negotiation.cc:299] Negotiation successful
0925 13:11:29.360009 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260925 13:11:29.360802 23181 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:11:29.361116 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.360999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:38457 (local address 127.0.0.1:51276)
0925 13:11:29.361045 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.361090 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:38457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 13:11:29.361279 23132 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.206.124:38457: connect: Connection refused (error 111)
I20260925 13:11:29.361687 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.361601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:38457 (local address 127.0.0.1:51280)
0925 13:11:29.361650 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.361672 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:38457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 13:11:29.361809 23181 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:38457: Network error: Client connection negotiation failed: client connection to 127.0.206.124:38457: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.124.keytab.
Entry for principal kudu/127.0.206.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.124.keytab.
I20260925 13:11:29.371012 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.124:38457
--webserver_interface=127.0.206.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.124.keytab
--principal=kudu/127.0.206.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:41911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:38555,127.0.206.125:41079,127.0.206.124:38457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:11:29.409619 23106 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } attempt: 3
I20260925 13:11:29.411015 23181 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } attempt: 1
I20260925 13:11:29.411388 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.411240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:38457 (local address 127.0.0.1:51288)
0925 13:11:29.411314 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.411355 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:38457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260925 13:11:29.411790 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.411691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:38457 (local address 127.0.0.1:51304)
0925 13:11:29.411740 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.411769 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:38457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 13:11:29.411912 23181 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:38457: Network error: Client connection negotiation failed: client connection to 127.0.206.124:38457: connect: Connection refused (error 111)
Sep 25 13:11:29 dist-test-slave-chkf krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
I20260925 13:11:29.416680 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.409986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:41079 (local address 127.0.0.1:43036)
0925 13:11:29.410043 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.410057 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:29.410117 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:29.410246 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:29.410247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:29.410422 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:29.410492 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:29.410494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.411290 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.411291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:29.412098 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:29.412101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.412496 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.412496 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:29.412569 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.413010 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:29.413016 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:29.414169 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:29.415829 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:29.415830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:29.415831 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:29.415991 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:29.416178 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:29.416178 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:29.416178 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:29.416226 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:29.416468 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:29.416469 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:29.416504 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 13:11:29.416553 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 13:11:29.416619 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 13:11:29.416719 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.410050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43036 (local address 127.0.206.125:41079)
0925 13:11:29.410108 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.410110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.410127 (+ 17us) server_negotiation.cc:407] Connection header received
0925 13:11:29.410177 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.410178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.410187 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.410237 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.410563 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.411189 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.412194 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.412411 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.412486 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.414248 (+ 1762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:29.414249 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:29.414250 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:29.414257 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:29.415744 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.416055 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.416056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.416056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.416114 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.416283 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.416284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.416284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.416408 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:29.416473 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:29.416575 (+ 102us) server_negotiation.cc:299] Negotiation successful
0925 13:11:29.416645 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 13:11:29.417101 23106 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:11:29.417387 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.417306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:38457 (local address 127.0.0.1:51310)
0925 13:11:29.417337 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.417365 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:38457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260925 13:11:29.417704 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.417623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:38457 (local address 127.0.0.1:51316)
0925 13:11:29.417667 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.417690 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:38457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 13:11:29.417809 23106 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:38457: Network error: Client connection negotiation failed: client connection to 127.0.206.124:38457: connect: Connection refused (error 111)
I20260925 13:11:29.448972 23106 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } attempt: 1
I20260925 13:11:29.449414 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.449276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:38457 (local address 127.0.0.1:51328)
0925 13:11:29.449340 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.449382 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:38457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 13:11:29.449761 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.449653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:38457 (local address 127.0.0.1:51336)
0925 13:11:29.449707 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.449738 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:38457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 13:11:29.449864 23106 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:38457: Network error: Client connection negotiation failed: client connection to 127.0.206.124:38457: connect: Connection refused (error 111)
W20260925 13:11:29.453940 23188 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 13:11:29.454123 23188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:29.454175 23188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:29.454207 23188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:29.455363 23188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:29.455402 23188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:29.455417 23188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:29.455461 23188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:29.455483 23188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:29.456923 23188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:38555,127.0.206.125:41079,127.0.206.124:38457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/krb5kdc/kudu_127.0.206.124.keytab
--principal=kudu/127.0.206.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.124:38457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.206.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:29.457158 23188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:29.457378 23188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:29.462001 23193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:29.462019 23194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:29.461998 23196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:29.462215 23188 server_base.cc:1034] running on GCE node
I20260925 13:11:29.462419 23188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:29.462644 23188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:29.463781 23188 hybrid_clock.cc:648] HybridClock initialized: now 1790341889463767 us; error 28 us; skew 500 ppm
Sep 25 13:11:29 dist-test-slave-chkf krb5kdc[23025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:29.466099 23188 init.cc:377] Logged in from keytab as kudu/127.0.206.124@KRBTEST.COM (short username kudu)
I20260925 13:11:29.466552 23188 webserver.cc:460] Webserver started at http://127.0.206.124:33741/ using document root <none> and password file <none>
I20260925 13:11:29.466825 23188 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:29.466871 23188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:29.467013 23188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:29.468019 23188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-2/data/instance:
uuid: "06e872eecba147448a79fa30c7f5b777"
format_stamp: "Formatted at 2026-09-25 13:11:29 on dist-test-slave-chkf"
server_key: "9646b506f11c1a9e6e74f6b71fd28a19"
server_key_iv: "082087504cde863068ff1df6cd9b9e7c"
server_key_version: "encryptionkey@0"
I20260925 13:11:29.468369 23188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-2/wal/instance:
uuid: "06e872eecba147448a79fa30c7f5b777"
format_stamp: "Formatted at 2026-09-25 13:11:29 on dist-test-slave-chkf"
server_key: "9646b506f11c1a9e6e74f6b71fd28a19"
server_key_iv: "082087504cde863068ff1df6cd9b9e7c"
server_key_version: "encryptionkey@0"
I20260925 13:11:29.469617 23188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 13:11:29.470304 23203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:29.470476 23188 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:11:29.470556 23188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-2/wal
uuid: "06e872eecba147448a79fa30c7f5b777"
format_stamp: "Formatted at 2026-09-25 13:11:29 on dist-test-slave-chkf"
server_key: "9646b506f11c1a9e6e74f6b71fd28a19"
server_key_iv: "082087504cde863068ff1df6cd9b9e7c"
server_key_version: "encryptionkey@0"
I20260925 13:11:29.470604 23188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:29.480149 23181 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } attempt: 2
I20260925 13:11:29.480530 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.480402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:38457 (local address 127.0.0.1:51350)
0925 13:11:29.480460 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.480497 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:38457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 13:11:29.480871 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.480788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:38457 (local address 127.0.0.1:51354)
0925 13:11:29.480829 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.480853 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:38457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 13:11:29.480986 23181 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:38457: Network error: Client connection negotiation failed: client connection to 127.0.206.124:38457: connect: Connection refused (error 111)
I20260925 13:11:29.494349 23188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:29.496214 23188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:29.496376 23188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:29.500274 23188 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.124:38457
I20260925 13:11:29.500344 23255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.124:38457 every 8 connection(s)
I20260925 13:11:29.500670 23188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-2/data/info.pb
I20260925 13:11:29.501616 23256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:29.503114 23256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:11:29.505103 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 23188
I20260925 13:11:29.505215 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790341627249256-825-0/minicluster-data/master-2/wal/instance
I20260925 13:11:29.505406 825 external_mini_cluster.cc:1428] Setting key bc6c9f2cdb3630b4445edc9d35f8a033
Sep 25 13:11:29 dist-test-slave-chkf krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
Sep 25 13:11:29 dist-test-slave-chkf krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260925 13:11:29.512957 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.503733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49634 (local address 127.0.206.126:38555)
0925 13:11:29.503809 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.503811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.503986 (+ 175us) server_negotiation.cc:407] Connection header received
0925 13:11:29.504464 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.504466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.504477 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.504542 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.505125 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.505754 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.507142 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.507433 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.507506 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.509889 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:29.509890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:29.509891 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:29.509900 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:29.511432 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.511871 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.511872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.511872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.511930 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.512224 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.512225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.512225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.512367 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:29.512437 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:29.512779 (+ 342us) server_negotiation.cc:299] Negotiation successful
0925 13:11:29.512843 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 13:11:29.513127 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.503697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38555 (local address 127.0.0.1:49634)
0925 13:11:29.503877 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.503909 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:29.504361 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:29.504612 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:29.504615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:29.504846 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:29.505023 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:29.505026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.505848 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.505849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:29.507067 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:29.507070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.507545 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.507547 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:29.507692 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.508181 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:29.508191 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:29.509796 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:29.511513 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:29.511519 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:29.511520 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:29.511794 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:29.512000 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:29.512001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:29.512001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:29.512150 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:29.512454 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:29.512461 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:29.512629 (+ 168us) client_negotiation.cc:769] Sending connection context
0925 13:11:29.512769 (+ 140us) client_negotiation.cc:240] Negotiation successful
0925 13:11:29.512923 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
I20260925 13:11:29.513615 23256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:11:29.515085 23260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.505773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51358 (local address 127.0.206.124:38457)
0925 13:11:29.506098 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.506101 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.506115 (+ 14us) server_negotiation.cc:407] Connection header received
0925 13:11:29.506131 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.506132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.506146 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.506205 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.506972 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.507897 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.508967 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.509437 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.509512 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.512051 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:29.512058 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:29.512058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:29.512064 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:29.513949 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.514350 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.514352 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.514353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.514408 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.514606 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.514606 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.514607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.514769 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:29.514879 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:29.514969 (+ 90us) server_negotiation.cc:299] Negotiation successful
0925 13:11:29.515023 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":118,"threads_started":1}
Sep 25 13:11:29 dist-test-slave-chkf krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
Sep 25 13:11:29 dist-test-slave-chkf krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:11:29.520627 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.513851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:41079 (local address 127.0.0.1:43044)
0925 13:11:29.513967 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.513978 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:29.514023 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:29.514147 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:29.514147 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:29.514242 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:29.514268 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:29.514269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.515341 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.515342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:29.515927 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:29.515929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.516325 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.516326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:29.516383 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.516771 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:29.516774 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:29.517835 (+ 1061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:29.519587 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:29.519588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:29.519589 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:29.519806 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:29.519988 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:29.519989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:29.519989 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:29.520046 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:29.520320 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:29.520321 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:29.520364 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 13:11:29.520454 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 13:11:29.520540 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260925 13:11:29.520628 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.513920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43044 (local address 127.0.206.125:41079)
0925 13:11:29.513961 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.513963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.514030 (+ 67us) server_negotiation.cc:407] Connection header received
0925 13:11:29.514075 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.514076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.514087 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.514145 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.514355 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.515065 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.516023 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.516236 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.516324 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.517913 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:29.517914 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:29.517915 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:29.517920 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:29.519509 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.519882 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.519883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.519883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.519928 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.520123 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.520124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.520124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.520248 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:29.520327 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:29.520479 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 13:11:29.520545 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 13:11:29.521283 23256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:11:29.523319 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.515825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49638 (local address 127.0.206.126:38555)
0925 13:11:29.515864 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.515866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.515873 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:11:29.515894 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.515895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.515904 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.515957 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.516400 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.516988 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.517683 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.517986 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.518036 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.520481 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:29.520483 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:29.520483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:29.520490 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:29.522207 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.522518 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.522519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.522519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.522569 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.522788 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.522789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.522789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.522892 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:29.523044 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:29.523089 (+ 45us) server_negotiation.cc:299] Negotiation successful
0925 13:11:29.523211 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 25 13:11:29 dist-test-slave-chkf krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260925 13:11:29.524999 23106 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } attempt: 2
Sep 25 13:11:29 dist-test-slave-chkf krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
I20260925 13:11:29.528618 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.521630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:38457 (local address 127.0.0.1:51368)
0925 13:11:29.521679 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.521688 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:29.521722 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:29.521830 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:29.521830 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:29.521955 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:29.521989 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:29.521990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.522548 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.522549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:29.523068 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:29.523070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.523564 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.523566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:29.523627 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.524115 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:29.524120 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:29.525274 (+ 1154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:29.527612 (+ 2338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:29.527612 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:29.527613 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:29.527834 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:29.528049 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:29.528049 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:29.528050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:29.528106 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:29.528354 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:29.528354 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:29.528389 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 13:11:29.528435 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 13:11:29.528516 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260925 13:11:29.528946 23260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.521680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51368 (local address 127.0.206.124:38457)
0925 13:11:29.521712 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.521713 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.521721 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:11:29.521773 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.521773 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.521781 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.521825 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.522055 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.522472 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.523155 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.523490 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.523544 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.525352 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:29.525353 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:29.525354 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:29.525360 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:29.527517 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.527910 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.527911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.527911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.527967 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.528173 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.528173 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.528173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.528285 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:29.528787 (+ 502us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:29.528826 (+ 39us) server_negotiation.cc:299] Negotiation successful
0925 13:11:29.528891 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Sep 25 13:11:29 dist-test-slave-chkf krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260925 13:11:29.530526 23256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777: Bootstrap starting.
I20260925 13:11:29.531293 23256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:29.531589 23256 log.cc:824] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:29.532354 23256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777: No bootstrap required, opened a new log
I20260925 13:11:29.532958 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.525217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:38457 (local address 127.0.0.1:51378)
0925 13:11:29.525265 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.525274 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:29.525309 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:29.525545 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:29.525546 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:29.525677 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:29.525732 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:29.525734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.526350 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.526351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:29.527194 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:29.527197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.528062 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.528064 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:29.528127 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.528519 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:29.528524 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:29.529838 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:29.531922 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:29.531923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:29.531923 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:29.532131 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:29.532330 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:29.532331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:29.532332 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:29.532386 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:29.532648 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:29.532649 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:29.532696 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 13:11:29.532774 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 13:11:29.532863 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 13:11:29.533003 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.525282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51378 (local address 127.0.206.124:38457)
0925 13:11:29.525426 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.525428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.525433 (+ 5us) server_negotiation.cc:407] Connection header received
0925 13:11:29.525448 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.525449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.525457 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.525516 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.525797 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.526269 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.527301 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.528000 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.528062 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.530131 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:29.530132 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:29.530132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:29.530138 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:29.531835 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.532218 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.532219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.532220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.532264 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.532455 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.532456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.532456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.532570 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:29.532643 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:29.532810 (+ 167us) server_negotiation.cc:299] Negotiation successful
0925 13:11:29.532881 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260925 13:11:29.533967 23256 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } } peers { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } } peers { permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } }
I20260925 13:11:29.534102 23256 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06e872eecba147448a79fa30c7f5b777, State: Initialized, Role: FOLLOWER
I20260925 13:11:29.534197 23256 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } } peers { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } } peers { permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } }
I20260925 13:11:29.534447 23263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } } peers { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } } peers { permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } } }
I20260925 13:11:29.534503 23256 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:29.534602 23106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4: Bootstrap starting.
I20260925 13:11:29.534768 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.523923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43060 (local address 127.0.206.125:41079)
0925 13:11:29.523955 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.523957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.524004 (+ 47us) server_negotiation.cc:407] Connection header received
0925 13:11:29.524073 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.524074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.524084 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.524137 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.524683 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.525546 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.526264 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.526466 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.526548 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.529197 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:29.529198 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:29.529198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:29.529203 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:29.530814 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.531190 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.531191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.531192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.531248 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.531516 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.531517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.531517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.531684 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:29.531825 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:29.534557 (+ 2732us) server_negotiation.cc:299] Negotiation successful
0925 13:11:29.534633 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 13:11:29.535353 23106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:29.535715 23106 log.cc:824] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:29.535917 23263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 13:11:29.536368 23274 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:11:29.536419 23274 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:11:29.536712 23106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4: No bootstrap required, opened a new log
I20260925 13:11:29.538595 23106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } } peers { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } } peers { permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } }
I20260925 13:11:29.538762 23106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a4531c5265f4ab499e97ef13675ebb4, State: Initialized, Role: FOLLOWER
I20260925 13:11:29.536705 23188 master_runner.cc:186] Error getting master registration for 127.0.206.126:38555: OK, instance_id { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" instance_seqno: 1790341889183022 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 13:11:29.538877 23188 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 13:11:29.538843 23106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } } peers { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } } peers { permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } }
I20260925 13:11:29.539142 23106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:29.539119 23277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } } peers { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } } peers { permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } } }
I20260925 13:11:29.539297 23277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:29.540462 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260925 13:11:29.540745 23288 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:11:29.540830 23288 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:11:29.541451 23080 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7a4531c5265f4ab499e97ef13675ebb4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49638
I20260925 13:11:29.541565 23080 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 13:11:29.541620 23080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:29.542538 23080 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } } peers { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } } peers { permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } }
I20260925 13:11:29.542807 23080 leader_election.cc:288] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [CANDIDATE]: Term 1 election: Requested vote from peers 0f7eb1f0df684fdf99a2c8d7a49438fe (127.0.206.125:41079), 06e872eecba147448a79fa30c7f5b777 (127.0.206.124:38457)
I20260925 13:11:29.541590 23038 master_runner.cc:186] Error getting master registration for 127.0.206.125:41079: OK, instance_id { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" instance_seqno: 1790341889337781 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 13:11:29.543828 23038 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 13:11:29.544165 23230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7a4531c5265f4ab499e97ef13675ebb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "06e872eecba147448a79fa30c7f5b777"
I20260925 13:11:29.544296 23230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:29.545737 23230 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a4531c5265f4ab499e97ef13675ebb4 in term 1.
I20260925 13:11:29.545984 23057 leader_election.cc:302] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06e872eecba147448a79fa30c7f5b777, 7a4531c5265f4ab499e97ef13675ebb4; no voters:
I20260925 13:11:29.546113 23277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:29.546271 23277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [term 1 LEADER]: Becoming Leader. State: Replica: 7a4531c5265f4ab499e97ef13675ebb4, State: Running, Role: LEADER
I20260925 13:11:29.546350 23277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } } peers { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } } peers { permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } }
I20260925 13:11:29.546868 23297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a4531c5265f4ab499e97ef13675ebb4. Latest consensus state: current_term: 1 leader_uuid: "7a4531c5265f4ab499e97ef13675ebb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } } peers { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } } peers { permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } } }
I20260925 13:11:29.546983 23297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:29.547231 23299 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:29.547441 23299 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:29.548918 23230 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777 [term 1 FOLLOWER]: Refusing update from remote peer 7a4531c5265f4ab499e97ef13675ebb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:29.549253 23297 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:29.550452 23263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a4531c5265f4ab499e97ef13675ebb4. Latest consensus state: current_term: 1 leader_uuid: "7a4531c5265f4ab499e97ef13675ebb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } } peers { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } } peers { permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } } }
I20260925 13:11:29.550552 23263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:29.561390 23300 mvcc.cc:205] Tried to move back new op lower bound from 7333240379590131712 to 7333240379582586880. Current Snapshot: MvccSnapshot[applied={T|T < 7333240379590131712}]
I20260925 13:11:29.561465 23277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7a4531c5265f4ab499e97ef13675ebb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } } peers { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } } peers { permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } } }
I20260925 13:11:29.561546 23277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:29.562042 23263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7a4531c5265f4ab499e97ef13675ebb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } } peers { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } } peers { permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } } }
I20260925 13:11:29.562119 23263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:29.562602 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.544994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49650 (local address 127.0.206.126:38555)
0925 13:11:29.545029 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.545031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.554856 (+ 9825us) server_negotiation.cc:407] Connection header received
0925 13:11:29.554902 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.554903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.554914 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.554972 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.555842 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.556564 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.557582 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.557881 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.557946 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.559502 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:29.559503 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:29.559504 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:29.559510 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:29.561289 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.561795 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.561796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.561797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.561845 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.562035 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.562036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.562036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.562148 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:29.562248 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:29.562446 (+ 198us) server_negotiation.cc:299] Negotiation successful
0925 13:11:29.562516 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:11:29.563688 23301 mvcc.cc:205] Tried to move back new op lower bound from 7333240379590131712 to 7333240379582586880. Current Snapshot: MvccSnapshot[applied={T|T < 7333240379590131712}]
I20260925 13:11:29.571936 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.545197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51388 (local address 127.0.206.124:38457)
0925 13:11:29.545230 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.545232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.559128 (+ 13896us) server_negotiation.cc:407] Connection header received
0925 13:11:29.559864 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.559865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.559878 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.559931 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.563794 (+ 3863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.564525 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.565513 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.565836 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.565875 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.567721 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:29.567722 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:29.567722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:29.567729 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:29.569815 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.571072 (+ 1257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.571073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.571074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.571125 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.571322 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.571323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.571336 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.571457 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:29.571552 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:29.571774 (+ 222us) server_negotiation.cc:299] Negotiation successful
0925 13:11:29.571853 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:11:29.572160 23181 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } attempt: 3
I20260925 13:11:29.572775 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.546022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43070 (local address 127.0.206.125:41079)
0925 13:11:29.546078 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.546080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.559039 (+ 12959us) server_negotiation.cc:407] Connection header received
0925 13:11:29.559477 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.559478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.559489 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.559519 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.563089 (+ 3570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.563817 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.565689 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.566004 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.566043 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.568484 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:29.568485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:29.568501 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:29.568508 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:29.570507 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.571314 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.571316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.571316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.571403 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.571773 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.571774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.571774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.571896 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:29.571983 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:29.572591 (+ 608us) server_negotiation.cc:299] Negotiation successful
0925 13:11:29.572666 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 13:11:29.573038 23299 catalog_manager.cc:1347] Generated new cluster ID: 630a7fb25dd144558b9fb8e512d17a77
I20260925 13:11:29.573084 23299 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 13:11:29 dist-test-slave-chkf krb5kdc[23025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260925 13:11:29.580761 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.572388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:38457 (local address 127.0.0.1:51390)
0925 13:11:29.572461 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:29.572470 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:29.572505 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:29.572715 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:29.572716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:29.572886 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:29.572924 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:29.572925 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.573803 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.573804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:29.574560 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:29.574562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.575085 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.575086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:29.575145 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.575602 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:29.575608 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:29.577071 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:29.579441 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:29.579442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:29.579442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:29.579657 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:29.579993 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:29.579994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:29.579994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:29.580049 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:29.580462 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:29.580463 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:29.580516 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 13:11:29.580573 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 13:11:29.580660 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260925 13:11:29.581485 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:29.572569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51390 (local address 127.0.206.124:38457)
0925 13:11:29.572600 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.572602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.572609 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:11:29.572625 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.572626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.572635 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.572677 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.573003 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.573705 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.574647 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:29.574967 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:29.575031 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:29.577176 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:29.577177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:29.577178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:29.577184 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:29.579296 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.579759 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.579760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.579760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.579872 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:29.580127 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:29.580128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:29.580128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:29.580329 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:29.581146 (+ 817us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:29.581219 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 13:11:29.581345 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:11:29.583261 23181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe: Bootstrap starting.
I20260925 13:11:29.583927 23181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:29.584205 23181 log.cc:824] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:29.588531 23299 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:29.589201 23181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe: No bootstrap required, opened a new log
I20260925 13:11:29.589689 23299 catalog_manager.cc:1504] Loading token signing keys...
I20260925 13:11:29.591250 23181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } } peers { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } } peers { permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } }
I20260925 13:11:29.591372 23181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f7eb1f0df684fdf99a2c8d7a49438fe, State: Initialized, Role: FOLLOWER
I20260925 13:11:29.591441 23181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } } peers { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } } peers { permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } }
I20260925 13:11:29.591732 23181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:29.591800 23154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:29.592617 23154 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe [term 1 FOLLOWER]: Refusing update from remote peer 7a4531c5265f4ab499e97ef13675ebb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:29.592875 23297 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:29.593111 23155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7a4531c5265f4ab499e97ef13675ebb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe"
I20260925 13:11:29.593933 23155 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a4531c5265f4ab499e97ef13675ebb4 in term 1.
I20260925 13:11:29.595233 23319 mvcc.cc:205] Tried to move back new op lower bound from 7333240379750965248 to 7333240379582586880. Current Snapshot: MvccSnapshot[applied={T|T < 7333240379590131712}]
I20260925 13:11:29.598982 23306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a4531c5265f4ab499e97ef13675ebb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } } peers { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } } peers { permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } } }
I20260925 13:11:29.599110 23306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:29.599192 23306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a4531c5265f4ab499e97ef13675ebb4. Latest consensus state: current_term: 1 leader_uuid: "7a4531c5265f4ab499e97ef13675ebb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } } peers { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } } peers { permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } } }
I20260925 13:11:29.599284 23306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:29.599351 23306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7a4531c5265f4ab499e97ef13675ebb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } } peers { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } } peers { permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } } }
I20260925 13:11:29.599452 23306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe [sys.catalog]: This master's current role is: FOLLOWER
W20260925 13:11:29.600021 23317 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:11:29.600112 23317 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:11:29.623507 23297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7a4531c5265f4ab499e97ef13675ebb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a4531c5265f4ab499e97ef13675ebb4" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38555 } } peers { permanent_uuid: "0f7eb1f0df684fdf99a2c8d7a49438fe" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41079 } } peers { permanent_uuid: "06e872eecba147448a79fa30c7f5b777" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 38457 } } }
I20260925 13:11:29.623603 23297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:29.625635 23299 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a4531c5265f4ab499e97ef13675ebb4: Generated new TSK 0
I20260925 13:11:29.625802 23299 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 13:11:29.646836 23109 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:29.645365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49666 (local address 127.0.206.126:38555)
0925 13:11:29.645405 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.645407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.646185 (+ 778us) server_negotiation.cc:407] Connection header received
0925 13:11:29.646302 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.646303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.646313 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.646342 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.646775 (+ 433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49666: BlockingRecv error: recv got EOF from 127.0.0.1:49666 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260925 13:11:29.647795 23261 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:29.645671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51394 (local address 127.0.206.124:38457)
0925 13:11:29.645708 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.645710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.647056 (+ 1346us) server_negotiation.cc:407] Connection header received
0925 13:11:29.647291 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.647292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.647304 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.647333 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.647737 (+ 404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51394: BlockingRecv error: recv got EOF from 127.0.0.1:51394 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260925 13:11:29.648239 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:29.645959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43080 (local address 127.0.206.125:41079)
0925 13:11:29.645995 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.645996 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.647144 (+ 1148us) server_negotiation.cc:407] Connection header received
0925 13:11:29.647392 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.647392 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.647402 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.647429 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.648186 (+ 757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43080: BlockingRecv error: recv got EOF from 127.0.0.1:43080 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 13:11:29.651961 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:29.651270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43082 (local address 127.0.206.125:41079)
0925 13:11:29.651301 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.651302 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.651493 (+ 191us) server_negotiation.cc:407] Connection header received
0925 13:11:29.651593 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.651594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.651610 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.651638 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.651921 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43082: BlockingRecv error: recv got EOF from 127.0.0.1:43082 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 13:11:29.654767 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:29.654057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43088 (local address 127.0.206.125:41079)
0925 13:11:29.654085 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.654087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.654296 (+ 209us) server_negotiation.cc:407] Connection header received
0925 13:11:29.654384 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.654384 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.654393 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.654418 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.654682 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43088: BlockingRecv error: recv got EOF from 127.0.0.1:43088 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260925 13:11:29.658515 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:29.657803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43096 (local address 127.0.206.125:41079)
0925 13:11:29.657841 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.657842 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.658040 (+ 198us) server_negotiation.cc:407] Connection header received
0925 13:11:29.658132 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.658133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.658141 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.658166 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.658467 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43096: BlockingRecv error: recv got EOF from 127.0.0.1:43096 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260925 13:11:29.664227 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:29.663561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43098 (local address 127.0.206.125:41079)
0925 13:11:29.663590 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.663592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.663794 (+ 202us) server_negotiation.cc:407] Connection header received
0925 13:11:29.663882 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.663883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.663892 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.663919 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.664190 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43098: BlockingRecv error: recv got EOF from 127.0.0.1:43098 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260925 13:11:29.673946 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:29.673224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43110 (local address 127.0.206.125:41079)
0925 13:11:29.673266 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.673268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.673465 (+ 197us) server_negotiation.cc:407] Connection header received
0925 13:11:29.673552 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.673553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.673561 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.673606 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.673905 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43110: BlockingRecv error: recv got EOF from 127.0.0.1:43110 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 13:11:29.691867 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:29.691077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43114 (local address 127.0.206.125:41079)
0925 13:11:29.691110 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.691112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.691354 (+ 242us) server_negotiation.cc:407] Connection header received
0925 13:11:29.691454 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.691455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.691464 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.691493 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.691825 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43114: BlockingRecv error: recv got EOF from 127.0.0.1:43114 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 13:11:29.726146 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:29.725212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43118 (local address 127.0.206.125:41079)
0925 13:11:29.725266 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.725269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.725535 (+ 266us) server_negotiation.cc:407] Connection header received
0925 13:11:29.725645 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.725646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.725658 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.725688 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.726086 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43118: BlockingRecv error: recv got EOF from 127.0.0.1:43118 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 13:11:29.792279 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:29.791549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43132 (local address 127.0.206.125:41079)
0925 13:11:29.791614 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.791620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.791716 (+ 96us) server_negotiation.cc:407] Connection header received
0925 13:11:29.791785 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.791786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.791795 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.791839 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.792173 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43132: BlockingRecv error: recv got EOF from 127.0.0.1:43132 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260925 13:11:29.922309 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:29.921475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43134 (local address 127.0.206.125:41079)
0925 13:11:29.921664 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:29.921666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:29.921717 (+ 51us) server_negotiation.cc:407] Connection header received
0925 13:11:29.921749 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:29.921760 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:29.921801 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:29.921867 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:29.922246 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43134: BlockingRecv error: recv got EOF from 127.0.0.1:43134 (error 108)
Metrics: {"server-negotiator.queue_time_us":152}
W20260925 13:11:30.180105 23185 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:30.179462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43148 (local address 127.0.206.125:41079)
0925 13:11:30.179531 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:30.179533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:30.179544 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:11:30.179600 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:30.179601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:30.179619 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:30.179684 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:30.180021 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43148: BlockingRecv error: recv got EOF from 127.0.0.1:43148 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 13:11:30.536818 23274 catalog_manager.cc:1259] Loaded cluster ID: 630a7fb25dd144558b9fb8e512d17a77
I20260925 13:11:30.536886 23274 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777: loading cluster ID for follower catalog manager: success
I20260925 13:11:30.537638 23274 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777: acquiring CA information for follower catalog manager: success
I20260925 13:11:30.537881 23274 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 06e872eecba147448a79fa30c7f5b777: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 13:11:30.600550 23317 catalog_manager.cc:1259] Loaded cluster ID: 630a7fb25dd144558b9fb8e512d17a77
I20260925 13:11:30.600618 23317 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe: loading cluster ID for follower catalog manager: success
I20260925 13:11:30.601480 23317 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe: acquiring CA information for follower catalog manager: success
I20260925 13:11:30.601804 23317 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0f7eb1f0df684fdf99a2c8d7a49438fe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 13:11:30.696053 23389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:30.693308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53446 (local address 127.0.206.125:41079)
0925 13:11:30.693427 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:30.693429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:30.693440 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:11:30.693519 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:30.693520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:30.693598 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:30.693686 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:30.693858 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:30.694367 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:30.695223 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:30.695445 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:30.695523 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:30.695698 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:30.695820 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:30.695911 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:30.695936 (+ 25us) server_negotiation.cc:299] Negotiation successful
0925 13:11:30.695958 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":48,"threads_started":1}
I20260925 13:11:30.700073 23395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:30.697761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48806 (local address 127.0.206.124:38457)
0925 13:11:30.697876 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:30.697878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:30.697902 (+ 24us) server_negotiation.cc:407] Connection header received
0925 13:11:30.698007 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:30.698008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:30.698054 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:30.698135 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:30.698286 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:30.698804 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:30.699479 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:30.699656 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:30.699702 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:30.699822 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:30.699908 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:30.699974 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:30.700004 (+ 30us) server_negotiation.cc:299] Negotiation successful
0925 13:11:30.700019 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":51,"threads_started":1}
I20260925 13:11:30.704527 23389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:30.701725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53452 (local address 127.0.206.125:41079)
0925 13:11:30.701761 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:30.701763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:30.701897 (+ 134us) server_negotiation.cc:407] Connection header received
0925 13:11:30.701948 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:30.701949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:30.701959 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:30.702017 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:30.702147 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:30.702889 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:30.703697 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:30.704003 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:30.704062 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:30.704227 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:30.704266 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:30.704312 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:30.704416 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 13:11:30.704438 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:11:30.704667 23395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:30.701876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48818 (local address 127.0.206.124:38457)
0925 13:11:30.701915 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:30.701917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:30.702004 (+ 87us) server_negotiation.cc:407] Connection header received
0925 13:11:30.702119 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:30.702120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:30.702133 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:30.702200 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:30.702292 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:30.702836 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:30.703803 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:30.704093 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:30.704153 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:30.704291 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:30.704323 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:30.704375 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:30.704471 (+ 96us) server_negotiation.cc:299] Negotiation successful
0925 13:11:30.704496 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 13:11:30.705060 23402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:30.701609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33592 (local address 127.0.206.126:38555)
0925 13:11:30.701983 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:30.701985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:30.701992 (+ 7us) server_negotiation.cc:407] Connection header received
0925 13:11:30.702031 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:30.702032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:30.702103 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:30.702181 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:30.702350 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:30.703104 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:30.704050 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:30.704387 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:30.704444 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:30.704639 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:30.704735 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:30.704799 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:30.704945 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 13:11:30.704973 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":47,"threads_started":1}
I20260925 13:11:30.706859 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 23038
I20260925 13:11:30.712328 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 23113
I20260925 13:11:30.717285 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 23188
2026-09-25T13:11:30Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2805 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 13:11:30 dist-test-slave-chkf krb5kdc[23406](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 13:11:30 dist-test-slave-chkf krb5kdc[23406](info): set up 2 sockets
Sep 25 13:11:30 dist-test-slave-chkf krb5kdc[23406](info): commencing operation
krb5kdc: starting...
W20260925 13:11:31.738898 825 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/test-user.keytab.
Sep 25 13:11:31 dist-test-slave-chkf krb5kdc[23406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T13:11:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T13:11:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260925 13:11:31.849144 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:45443
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:45443,127.0.206.125:41537,127.0.206.124:37153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 13:11:31.927443 23419 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 13:11:31.927665 23419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:31.927683 23419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:31.927702 23419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:31.928880 23419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:31.928925 23419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:31.928939 23419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:31.928952 23419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:31.928964 23419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:31.930449 23419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:45443,127.0.206.125:41537,127.0.206.124:37153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:45443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:31.930694 23419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:31.930956 23419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:31.935695 23425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:31.935722 23424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:31.935885 23427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:31.935928 23419 server_base.cc:1034] running on GCE node
I20260925 13:11:31.936146 23419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:31.936391 23419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:31.937530 23419 hybrid_clock.cc:648] HybridClock initialized: now 1790341891937517 us; error 28 us; skew 500 ppm
Sep 25 13:11:31 dist-test-slave-chkf krb5kdc[23406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341891, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:31.944775 23419 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260925 13:11:31.945246 23419 webserver.cc:460] Webserver started at http://127.0.206.126:40741/ using document root <none> and password file <none>
I20260925 13:11:31.945480 23419 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:31.945549 23419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:31.945705 23419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:31.946810 23419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-0/data/instance:
uuid: "b11533fa00c2407ea1754b8a93dabe80"
format_stamp: "Formatted at 2026-09-25 13:11:31 on dist-test-slave-chkf"
server_key: "ba966bb5f6bdc570995757d9fc4a4c03"
server_key_iv: "fae10c014d613c7157a778b1acca9e68"
server_key_version: "encryptionkey@0"
I20260925 13:11:31.947173 23419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-0/wal/instance:
uuid: "b11533fa00c2407ea1754b8a93dabe80"
format_stamp: "Formatted at 2026-09-25 13:11:31 on dist-test-slave-chkf"
server_key: "ba966bb5f6bdc570995757d9fc4a4c03"
server_key_iv: "fae10c014d613c7157a778b1acca9e68"
server_key_version: "encryptionkey@0"
I20260925 13:11:31.948486 23419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 13:11:31.949200 23434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:31.949388 23419 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 13:11:31.949486 23419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-0/wal
uuid: "b11533fa00c2407ea1754b8a93dabe80"
format_stamp: "Formatted at 2026-09-25 13:11:31 on dist-test-slave-chkf"
server_key: "ba966bb5f6bdc570995757d9fc4a4c03"
server_key_iv: "fae10c014d613c7157a778b1acca9e68"
server_key_version: "encryptionkey@0"
I20260925 13:11:31.949563 23419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:31.959542 23419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:31.961221 23419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:31.961382 23419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:31.965240 23419 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:45443
I20260925 13:11:31.965277 23486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:45443 every 8 connection(s)
I20260925 13:11:31.965610 23419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-0/data/info.pb
I20260925 13:11:31.966634 23487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:31.968165 23487 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:11:31.973280 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 23419
I20260925 13:11:31.973366 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-0/wal/instance
I20260925 13:11:31.973562 825 external_mini_cluster.cc:1428] Setting key 90bc419fdc97ef5ab37d7df3d6606629
Sep 25 13:11:31 dist-test-slave-chkf krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341891, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:11:31.977778 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:31.968770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45443 (local address 127.0.0.1:47380)
0925 13:11:31.968922 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:31.968963 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:31.969391 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:31.969633 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:31.969635 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:31.969869 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:31.970018 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:31.970022 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:31.971016 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:31.971016 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:31.971665 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:31.971667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:31.972113 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:31.972114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:31.972164 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:31.972575 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:31.972584 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:31.974210 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:31.976365 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:31.976371 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:31.976372 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:31.976558 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:31.976823 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:31.976823 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:31.976824 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:31.976898 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:31.977356 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:31.977363 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:31.977438 (+ 75us) client_negotiation.cc:769] Sending connection context
0925 13:11:31.977510 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 13:11:31.977624 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260925 13:11:31.977857 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:31.968831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47380 (local address 127.0.206.126:45443)
0925 13:11:31.968975 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:31.968979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:31.969054 (+ 75us) server_negotiation.cc:407] Connection header received
0925 13:11:31.969511 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:31.969513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:31.969529 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:31.969607 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:31.970099 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:31.970940 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:31.971734 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:31.972025 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:31.972099 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:31.974314 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:31.974321 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:31.974322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:31.974327 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:31.976282 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:31.976645 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:31.976648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:31.976648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:31.976754 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:31.976962 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:31.976963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:31.976964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:31.977271 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:31.977363 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:31.977578 (+ 215us) server_negotiation.cc:299] Negotiation successful
0925 13:11:31.977710 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260925 13:11:31.978559 23487 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:11:31.978911 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:31.978803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:41537 (local address 127.0.0.1:42250)
0925 13:11:31.978851 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:31.978895 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 13:11:31.979065 23438 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41537: connect: Connection refused (error 111)
I20260925 13:11:31.979496 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:31.979411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:41537 (local address 127.0.0.1:42260)
0925 13:11:31.979457 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:31.979481 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 13:11:31.979631 23487 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.125:41537: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41537: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.0.206.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/kudu_127.0.206.125.keytab.
Entry for principal kudu/127.0.206.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/kudu_127.0.206.125.keytab.
I20260925 13:11:31.999038 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.125:41537
--webserver_interface=127.0.206.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/kudu_127.0.206.125.keytab
--principal=kudu/127.0.206.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:45443,127.0.206.125:41537,127.0.206.124:37153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:11:32.028880 23487 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } attempt: 1
I20260925 13:11:32.029345 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.029209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:41537 (local address 127.0.0.1:42274)
0925 13:11:32.029274 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.029316 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 13:11:32.029703 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.029601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:41537 (local address 127.0.0.1:42282)
0925 13:11:32.029654 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.029678 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260925 13:11:32.029803 23487 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.125:41537: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41537: connect: Connection refused (error 111)
W20260925 13:11:32.077267 23494 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 13:11:32.077494 23494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:32.077523 23494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:32.077564 23494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 13:11:32.078640 23494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:32.078706 23494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:32.078758 23494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:32.078789 23494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:32.078814 23494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:32.080247 23494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:45443,127.0.206.125:41537,127.0.206.124:37153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/kudu_127.0.206.125.keytab
--principal=kudu/127.0.206.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.125:41537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.206.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:32.080504 23494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:32.080713 23494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:32.085484 23502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:32.085464 23500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:32.085543 23494 server_base.cc:1034] running on GCE node
W20260925 13:11:32.085460 23499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:32.085878 23494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:32.086153 23494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:32.087291 23494 hybrid_clock.cc:648] HybridClock initialized: now 1790341892087279 us; error 28 us; skew 500 ppm
Sep 25 13:11:32 dist-test-slave-chkf krb5kdc[23406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341892, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:32.089695 23494 init.cc:377] Logged in from keytab as kudu/127.0.206.125@KRBTEST.COM (short username kudu)
I20260925 13:11:32.090081 23494 webserver.cc:460] Webserver started at http://127.0.206.125:43947/ using document root <none> and password file <none>
I20260925 13:11:32.090317 23494 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:32.090385 23494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:32.090533 23494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:32.091534 23494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-1/data/instance:
uuid: "c2ef96ab77204786949428aa4c670ff6"
format_stamp: "Formatted at 2026-09-25 13:11:32 on dist-test-slave-chkf"
server_key: "814693f21d45a6e549a445b27beae995"
server_key_iv: "fcd663677a41ad327b487410d5d16d1f"
server_key_version: "encryptionkey@0"
I20260925 13:11:32.091897 23494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-1/wal/instance:
uuid: "c2ef96ab77204786949428aa4c670ff6"
format_stamp: "Formatted at 2026-09-25 13:11:32 on dist-test-slave-chkf"
server_key: "814693f21d45a6e549a445b27beae995"
server_key_iv: "fcd663677a41ad327b487410d5d16d1f"
server_key_version: "encryptionkey@0"
I20260925 13:11:32.093134 23494 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:11:32.093891 23509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:32.094177 23494 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 13:11:32.094277 23494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-1/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-1/wal
uuid: "c2ef96ab77204786949428aa4c670ff6"
format_stamp: "Formatted at 2026-09-25 13:11:32 on dist-test-slave-chkf"
server_key: "814693f21d45a6e549a445b27beae995"
server_key_iv: "fcd663677a41ad327b487410d5d16d1f"
server_key_version: "encryptionkey@0"
I20260925 13:11:32.094362 23494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:32.097934 23487 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } attempt: 2
I20260925 13:11:32.098292 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.098174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:41537 (local address 127.0.0.1:42286)
0925 13:11:32.098232 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.098268 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 13:11:32.098645 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.098554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:41537 (local address 127.0.0.1:42288)
0925 13:11:32.098597 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.098621 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 13:11:32.098748 23487 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.125:41537: Network error: Client connection negotiation failed: client connection to 127.0.206.125:41537: connect: Connection refused (error 111)
I20260925 13:11:32.108114 23494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:32.110320 23494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:32.110481 23494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:32.114265 23494 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.125:41537
I20260925 13:11:32.114327 23561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.125:41537 every 8 connection(s)
I20260925 13:11:32.114665 23494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-1/data/info.pb
I20260925 13:11:32.115634 23562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:32.117096 23562 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 13:11:32 dist-test-slave-chkf krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341892, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:11:32.123230 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 23494
I20260925 13:11:32.123353 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-1/wal/instance
I20260925 13:11:32.123593 825 external_mini_cluster.cc:1428] Setting key ab6cb9d8376f8ccf638e6f9851c0c3bf
I20260925 13:11:32.126190 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.117796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47382 (local address 127.0.206.126:45443)
0925 13:11:32.117879 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.117882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.118059 (+ 177us) server_negotiation.cc:407] Connection header received
0925 13:11:32.118575 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.118576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.118590 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.118643 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.119294 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.119790 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.120791 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.120995 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.121074 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.123083 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:32.123084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:32.123084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:32.123092 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:32.124584 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.125113 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.125114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.125114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.125172 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.125453 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.125454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.125454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.125559 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:32.125627 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.125941 (+ 314us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.126047 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 13:11:32.126355 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.117779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45443 (local address 127.0.0.1:47382)
0925 13:11:32.117939 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.117974 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:32.118467 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:32.118773 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:32.118777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:32.119009 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:32.119195 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:32.119199 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.119902 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.119904 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:32.120697 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:32.120699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.121092 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.121093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:32.121219 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.121629 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:32.121637 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:32.122987 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:32.124698 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:32.124704 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:32.124706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:32.125018 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:32.125246 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:32.125247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:32.125248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:32.125388 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:32.125629 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:32.125634 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:32.125798 (+ 164us) client_negotiation.cc:769] Sending connection context
0925 13:11:32.125924 (+ 126us) client_negotiation.cc:240] Negotiation successful
0925 13:11:32.126093 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260925 13:11:32.126870 23562 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 13:11:32 dist-test-slave-chkf krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341892, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.206.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.124@KRBTEST.COM" created.
I20260925 13:11:32.134375 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.127133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:41537 (local address 127.0.0.1:42304)
0925 13:11:32.127339 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.127350 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:32.127398 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:32.127540 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:32.127541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:32.127686 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:32.127732 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:32.127733 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.128551 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.128552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:32.129037 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:32.129039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.129518 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.129519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:32.129579 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.129989 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:32.129992 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:32.131144 (+ 1152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:32.133073 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:32.133074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:32.133074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:32.133237 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:32.133539 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:32.133540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:32.133540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:32.133614 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:32.133977 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:32.133977 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:32.134016 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 13:11:32.134105 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 13:11:32.134258 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 13:11:32.134512 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.127209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42304 (local address 127.0.206.125:41537)
0925 13:11:32.127350 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.127355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.127397 (+ 42us) server_negotiation.cc:407] Connection header received
0925 13:11:32.127453 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.127455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.127467 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.127539 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.127814 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.128456 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.129131 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.129432 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.129518 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.131248 (+ 1730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:32.131255 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:32.131255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:32.131262 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:32.132976 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.133343 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.133345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.133346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.133421 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.133729 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.133730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.133731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.133893 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:32.133998 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.134145 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.134312 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":113,"thread_start_us":92,"threads_started":1}
I20260925 13:11:32.135627 23562 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:11:32.135967 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.135854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:37153 (local address 127.0.0.1:55476)
0925 13:11:32.135902 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.135950 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 13:11:32.136113 23511 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.206.124:37153: connect: Connection refused (error 111)
I20260925 13:11:32.136492 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.136396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:37153 (local address 127.0.0.1:55486)
0925 13:11:32.136446 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.136469 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 13:11:32.136596 23562 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:37153: Network error: Client connection negotiation failed: client connection to 127.0.206.124:37153: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/kudu_127.0.206.124.keytab.
Entry for principal kudu/127.0.206.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/kudu_127.0.206.124.keytab.
I20260925 13:11:32.148072 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
/tmp/dist-test-taskG_Ts_H/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.124:37153
--webserver_interface=127.0.206.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/kudu_127.0.206.124.keytab
--principal=kudu/127.0.206.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:45443,127.0.206.125:41537,127.0.206.124:37153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 13:11:32.185806 23562 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } attempt: 1
I20260925 13:11:32.186301 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.186173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:37153 (local address 127.0.0.1:55496)
0925 13:11:32.186234 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.186274 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 13:11:32.186661 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.186578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:37153 (local address 127.0.0.1:55502)
0925 13:11:32.186620 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.186644 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 13:11:32.186793 23562 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:37153: Network error: Client connection negotiation failed: client connection to 127.0.206.124:37153: connect: Connection refused (error 111)
I20260925 13:11:32.189885 23487 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } attempt: 3
Sep 25 13:11:32 dist-test-slave-chkf krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341891, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
I20260925 13:11:32.196790 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.190206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42316 (local address 127.0.206.125:41537)
0925 13:11:32.190268 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.190270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.190297 (+ 27us) server_negotiation.cc:407] Connection header received
0925 13:11:32.190322 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.190323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.190334 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.190385 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.190726 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.191221 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.191947 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.192191 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.192279 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.194231 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:32.194233 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:32.194233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:32.194240 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:32.195650 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.196021 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.196022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.196023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.196079 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.196284 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.196285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.196285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.196397 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:32.196468 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.196627 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.196707 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 13:11:32.196794 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.190133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:41537 (local address 127.0.0.1:42316)
0925 13:11:32.190192 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.190209 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:32.190263 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:32.190396 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:32.190397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:32.190552 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:32.190616 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:32.190618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.191314 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.191315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:32.191861 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:32.191862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.192280 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.192281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:32.192362 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.192830 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:32.192834 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:32.194142 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:32.195762 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:32.195763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:32.195764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:32.195937 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:32.196146 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:32.196147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:32.196148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:32.196219 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:32.196474 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:32.196475 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:32.196518 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 13:11:32.196602 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 13:11:32.196699 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 13:11:32.197219 23487 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:11:32.197535 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.197441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:37153 (local address 127.0.0.1:55510)
0925 13:11:32.197490 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.197517 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 13:11:32.198160 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.198074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:37153 (local address 127.0.0.1:55518)
0925 13:11:32.198116 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.198139 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 13:11:32.198266 23487 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:37153: Network error: Client connection negotiation failed: client connection to 127.0.206.124:37153: connect: Connection refused (error 111)
W20260925 13:11:32.228708 23569 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 13:11:32.228946 23569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 13:11:32.228976 23569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 13:11:32.229017 23569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260925 13:11:32.229395 23487 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } attempt: 1
I20260925 13:11:32.229785 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.229653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:37153 (local address 127.0.0.1:55524)
0925 13:11:32.229724 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.229761 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260925 13:11:32.230093 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.230004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:37153 (local address 127.0.0.1:55538)
0925 13:11:32.230046 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.230069 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 13:11:32.230168 23487 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:37153: Network error: Client connection negotiation failed: client connection to 127.0.206.124:37153: connect: Connection refused (error 111)
W20260925 13:11:32.230322 23569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 13:11:32.230362 23569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 13:11:32.230377 23569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 13:11:32.230415 23569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 13:11:32.230443 23569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 13:11:32.231971 23569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:45443,127.0.206.125:41537,127.0.206.124:37153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/krb5kdc/kudu_127.0.206.124.keytab
--principal=kudu/127.0.206.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.124:37153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.206.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 12:18:27 UTC on ccycloud.quasar-mgemhg.root.comops.site
build id 4672476
I20260925 13:11:32.232194 23569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 13:11:32.232429 23569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 13:11:32.237047 23575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:32.237114 23569 server_base.cc:1034] running on GCE node
W20260925 13:11:32.237068 23574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 13:11:32.237049 23577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 13:11:32.237443 23569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 13:11:32.237740 23569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 13:11:32.238905 23569 hybrid_clock.cc:648] HybridClock initialized: now 1790341892238882 us; error 39 us; skew 500 ppm
Sep 25 13:11:32 dist-test-slave-chkf krb5kdc[23406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341892, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 13:11:32.241199 23569 init.cc:377] Logged in from keytab as kudu/127.0.206.124@KRBTEST.COM (short username kudu)
I20260925 13:11:32.241593 23569 webserver.cc:460] Webserver started at http://127.0.206.124:33015/ using document root <none> and password file <none>
I20260925 13:11:32.241830 23569 fs_manager.cc:359] Metadata directory not provided
I20260925 13:11:32.241875 23569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 13:11:32.242038 23569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 13:11:32.243114 23569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-2/data/instance:
uuid: "22cff55758cf49c4ac202c1edc1d4c0d"
format_stamp: "Formatted at 2026-09-25 13:11:32 on dist-test-slave-chkf"
server_key: "54f25aeccf7f8e174a5b49e8b638a739"
server_key_iv: "83ba8d5fffadc691fc31967dfc37fcab"
server_key_version: "encryptionkey@0"
I20260925 13:11:32.243480 23569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-2/wal/instance:
uuid: "22cff55758cf49c4ac202c1edc1d4c0d"
format_stamp: "Formatted at 2026-09-25 13:11:32 on dist-test-slave-chkf"
server_key: "54f25aeccf7f8e174a5b49e8b638a739"
server_key_iv: "83ba8d5fffadc691fc31967dfc37fcab"
server_key_version: "encryptionkey@0"
I20260925 13:11:32.244735 23569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 13:11:32.245383 23584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 13:11:32.245553 23569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 13:11:32.245622 23569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-2/data,/tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-2/wal
uuid: "22cff55758cf49c4ac202c1edc1d4c0d"
format_stamp: "Formatted at 2026-09-25 13:11:32 on dist-test-slave-chkf"
server_key: "54f25aeccf7f8e174a5b49e8b638a739"
server_key_iv: "83ba8d5fffadc691fc31967dfc37fcab"
server_key_version: "encryptionkey@0"
I20260925 13:11:32.245685 23569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 13:11:32.254945 23562 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } attempt: 2
I20260925 13:11:32.255311 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.255190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:37153 (local address 127.0.0.1:55542)
0925 13:11:32.255249 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.255283 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 13:11:32.255661 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.255576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:37153 (local address 127.0.0.1:55550)
0925 13:11:32.255620 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.255645 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 13:11:32.255752 23562 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:37153: Network error: Client connection negotiation failed: client connection to 127.0.206.124:37153: connect: Connection refused (error 111)
I20260925 13:11:32.258935 23569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 13:11:32.260654 23569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 13:11:32.260802 23569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 13:11:32.264631 23569 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.124:37153
I20260925 13:11:32.264678 23636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.124:37153 every 8 connection(s)
I20260925 13:11:32.265000 23569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-2/data/info.pb
I20260925 13:11:32.265936 23637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 13:11:32.267521 23637 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } has no permanent_uuid. Determining permanent_uuid...
I20260925 13:11:32.272578 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu as pid 23569
I20260925 13:11:32.272701 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskG_Ts_H/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790341627249256-825-0/minicluster-data/master-2/wal/instance
Sep 25 13:11:32 dist-test-slave-chkf krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341892, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260925 13:11:32.272910 825 external_mini_cluster.cc:1428] Setting key 7ed870c6e555a43d607163c29c128d13
I20260925 13:11:32.276161 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.268174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47396 (local address 127.0.206.126:45443)
0925 13:11:32.268246 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.268248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.268451 (+ 203us) server_negotiation.cc:407] Connection header received
0925 13:11:32.268884 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.268884 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.268893 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.268945 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.269481 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.269932 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.270891 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.271066 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.271118 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.273293 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:32.273294 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:32.273295 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:32.273302 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:32.274827 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.275239 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.275240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.275241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.275300 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.275552 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.275553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.275553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.275664 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:32.275738 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.275998 (+ 260us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.276076 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 13:11:32.276355 23639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.268158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45443 (local address 127.0.0.1:47396)
0925 13:11:32.268344 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.268377 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:32.268808 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:32.269015 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:32.269018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:32.269241 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:32.269415 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:32.269419 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.270003 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.270004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:32.270825 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:32.270827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.271143 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.271144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:32.271238 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.271634 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:32.271642 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:32.273158 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:32.274922 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:32.274929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:32.274931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:32.275148 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:32.275368 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:32.275368 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:32.275369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:32.275478 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:32.275731 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:32.275737 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:32.275869 (+ 132us) client_negotiation.cc:769] Sending connection context
0925 13:11:32.275975 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 13:11:32.276118 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":108,"threads_started":1}
I20260925 13:11:32.276868 23637 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 13:11:32 dist-test-slave-chkf krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
Sep 25 13:11:32 dist-test-slave-chkf krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341892, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
I20260925 13:11:32.281956 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.273387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55560 (local address 127.0.206.124:37153)
0925 13:11:32.273554 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.273559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.273584 (+ 25us) server_negotiation.cc:407] Connection header received
0925 13:11:32.273672 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.273674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.273686 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.273777 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.274381 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.275124 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.275884 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.276175 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.276258 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.279005 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:32.279012 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:32.279012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:32.279019 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:32.280677 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.281041 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.281044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.281044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.281098 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.281334 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.281335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.281335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.281503 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:32.281657 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.281734 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.281826 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":70,"threads_started":1}
I20260925 13:11:32.283648 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.277203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42320 (local address 127.0.206.125:41537)
0925 13:11:32.277247 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.277249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.277258 (+ 9us) server_negotiation.cc:407] Connection header received
0925 13:11:32.277284 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.277285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.277294 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.277366 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.277580 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.278200 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.279036 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.279222 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.279316 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.280866 (+ 1550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:32.280867 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:32.280868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:32.280874 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:32.282599 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.282974 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.282975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.282975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.283032 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.283183 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.283184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.283184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.283323 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:32.283393 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.283535 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.283590 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 13:11:32.283649 23639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.277121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:41537 (local address 127.0.0.1:42320)
0925 13:11:32.277176 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.277185 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:32.277220 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:32.277365 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:32.277366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:32.277471 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:32.277510 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:32.277511 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.278281 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.278282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:32.278940 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:32.278942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.279317 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.279318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:32.279389 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.279699 (+ 310us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:32.279703 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:32.280774 (+ 1071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:32.282660 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:32.282661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:32.282661 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:32.282903 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:32.283097 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:32.283098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:32.283098 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:32.283144 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:32.283382 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:32.283382 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:32.283410 (+ 28us) client_negotiation.cc:769] Sending connection context
0925 13:11:32.283485 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 13:11:32.283564 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 13:11:32.284017 23637 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 13:11:32 dist-test-slave-chkf krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
Sep 25 13:11:32 dist-test-slave-chkf krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341892, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260925 13:11:32.290314 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.282761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47406 (local address 127.0.206.126:45443)
0925 13:11:32.282802 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.282804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.282815 (+ 11us) server_negotiation.cc:407] Connection header received
0925 13:11:32.282838 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.282839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.282849 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.282916 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.283477 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.284115 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.285003 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.285199 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.285247 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.287700 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:32.287701 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:32.287701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:32.287707 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:32.289246 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.289552 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.289553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.289554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.289610 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.289823 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.289824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.289825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.289934 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:32.290103 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.290137 (+ 34us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.290208 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 13:11:32.290756 23639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.284199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:37153 (local address 127.0.0.1:55572)
0925 13:11:32.284271 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.284280 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:32.284321 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:32.284435 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:32.284436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:32.284535 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:32.284560 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:32.284561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.285328 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.285329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:32.286086 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:32.286089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.286443 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.286444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:32.286498 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.286912 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:32.286916 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:32.288247 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:32.289748 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:32.289749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:32.289749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:32.289900 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:32.290103 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:32.290103 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:32.290104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:32.290162 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:32.290451 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:32.290452 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:32.290487 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 13:11:32.290544 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 13:11:32.290633 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260925 13:11:32.290812 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.284233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55572 (local address 127.0.206.124:37153)
0925 13:11:32.284265 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.284266 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.284312 (+ 46us) server_negotiation.cc:407] Connection header received
0925 13:11:32.284357 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.284357 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.284366 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.284424 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.284627 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.285251 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.286176 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.286370 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.286422 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.288360 (+ 1938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:32.288361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:32.288361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:32.288367 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:32.289695 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.289984 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.289985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.289985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.290031 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.290231 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.290232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.290232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.290368 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:32.290429 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.290577 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.290679 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":16}
I20260925 13:11:32.292120 23637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d: Bootstrap starting.
I20260925 13:11:32.292897 23637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:32.293170 23637 log.cc:824] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:32.293944 23637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d: No bootstrap required, opened a new log
Sep 25 13:11:32 dist-test-slave-chkf krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
I20260925 13:11:32.295413 23637 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } } peers { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } } peers { permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } }
I20260925 13:11:32.295543 23637 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22cff55758cf49c4ac202c1edc1d4c0d, State: Initialized, Role: FOLLOWER
I20260925 13:11:32.295632 23637 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } } peers { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } } peers { permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } }
I20260925 13:11:32.295908 23643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } } peers { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } } peers { permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } } }
I20260925 13:11:32.295969 23637 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:32.296025 23643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d [sys.catalog]: This master's current role is: FOLLOWER
W20260925 13:11:32.297830 23654 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:11:32.297889 23654 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:11:32.298420 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.290992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42326 (local address 127.0.206.125:41537)
0925 13:11:32.291022 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.291024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.291034 (+ 10us) server_negotiation.cc:407] Connection header received
0925 13:11:32.291055 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.291056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.291064 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.291146 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.291656 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.292084 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.292740 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.292952 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.293031 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.295237 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:32.295239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:32.295239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:32.295246 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:32.296786 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.297223 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.297224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.297225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.297283 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.297734 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.297736 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.297736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.297863 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:32.297998 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.298241 (+ 243us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.298309 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 13:11:32.299887 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 13:11:32.298080 23569 master_runner.cc:186] Error getting master registration for 127.0.206.126:45443: OK, instance_id { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" instance_seqno: 1790341891964194 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 13:11:32.300832 23569 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 13:11:32.300617 23461 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b11533fa00c2407ea1754b8a93dabe80"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47406
I20260925 13:11:32.305306 23487 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } attempt: 2
Sep 25 13:11:32 dist-test-slave-chkf krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341891, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260925 13:11:32.312316 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.305606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55582 (local address 127.0.206.124:37153)
0925 13:11:32.305677 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.305679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.305687 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:11:32.305708 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.305709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.305718 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.305772 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.306045 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.306793 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.307440 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.307877 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.307992 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.309821 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:32.309822 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:32.309822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:32.309828 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:32.311187 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.311558 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.311559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.311559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.311615 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.311807 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.311808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.311808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.311955 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:32.312019 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.312164 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.312230 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 13:11:32.312335 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.305555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:37153 (local address 127.0.0.1:55582)
0925 13:11:32.305611 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.305620 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:32.305658 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:32.305776 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:32.305777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:32.305932 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:32.305971 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:32.305972 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.306871 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.306872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:32.307348 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:32.307350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.307996 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.307998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:32.308057 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.308470 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:32.308475 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:32.309732 (+ 1257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:32.311273 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:32.311274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:32.311274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:32.311484 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:32.311685 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:32.311686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:32.311686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:32.311746 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:32.312023 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:32.312024 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:32.312067 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 13:11:32.312142 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 13:11:32.312227 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260925 13:11:32.313545 23487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80: Bootstrap starting.
I20260925 13:11:32.314113 23487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:32.314416 23487 log.cc:824] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:32.315155 23487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80: No bootstrap required, opened a new log
I20260925 13:11:32.316397 23487 raft_consensus.cc:348] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } } peers { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } } peers { permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } }
I20260925 13:11:32.316522 23487 raft_consensus.cc:729] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b11533fa00c2407ea1754b8a93dabe80, State: Initialized, Role: FOLLOWER
I20260925 13:11:32.316589 23487 consensus_queue.cc:260] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } } peers { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } } peers { permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } }
I20260925 13:11:32.316861 23487 sys_catalog.cc:564] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:32.316891 23461 raft_consensus.cc:482] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 13:11:32.316933 23461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:32.317783 23461 raft_consensus.cc:504] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } } peers { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } } peers { permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } }
I20260925 13:11:32.318045 23461 leader_election.cc:288] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [CANDIDATE]: Term 1 election: Requested vote from peers c2ef96ab77204786949428aa4c670ff6 (127.0.206.125:41537), 22cff55758cf49c4ac202c1edc1d4c0d (127.0.206.124:37153)
I20260925 13:11:32.318504 23611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b11533fa00c2407ea1754b8a93dabe80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "22cff55758cf49c4ac202c1edc1d4c0d"
I20260925 13:11:32.318619 23611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d [term 0 FOLLOWER]: Advancing to term 1
W20260925 13:11:32.318998 23668 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 13:11:32.319094 23668 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 13:11:32.320554 23419 master_runner.cc:186] Error getting master registration for 127.0.206.125:41537: OK, instance_id { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" instance_seqno: 1790341892113229 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 13:11:32.322758 23419 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 13:11:32.323592 23611 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b11533fa00c2407ea1754b8a93dabe80 in term 1.
I20260925 13:11:32.323829 23436 leader_election.cc:302] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22cff55758cf49c4ac202c1edc1d4c0d, b11533fa00c2407ea1754b8a93dabe80; no voters:
I20260925 13:11:32.316839 23657 sys_catalog.cc:455] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } } peers { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } } peers { permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } } }
I20260925 13:11:32.324157 23657 sys_catalog.cc:458] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:32.324096 23676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 13:11:32.324348 23676 raft_consensus.cc:686] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [term 1 LEADER]: Becoming Leader. State: Replica: b11533fa00c2407ea1754b8a93dabe80, State: Running, Role: LEADER
I20260925 13:11:32.324496 23676 consensus_queue.cc:237] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } } peers { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } } peers { permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } }
I20260925 13:11:32.324939 23657 sys_catalog.cc:455] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b11533fa00c2407ea1754b8a93dabe80. Latest consensus state: current_term: 1 leader_uuid: "b11533fa00c2407ea1754b8a93dabe80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } } peers { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } } peers { permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } } }
I20260925 13:11:32.325151 23657 sys_catalog.cc:458] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:32.325362 23678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 13:11:32.325735 23678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 13:11:32.327271 23611 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d [term 1 FOLLOWER]: Refusing update from remote peer b11533fa00c2407ea1754b8a93dabe80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:32.327657 23676 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:32.328958 23643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b11533fa00c2407ea1754b8a93dabe80. Latest consensus state: current_term: 1 leader_uuid: "b11533fa00c2407ea1754b8a93dabe80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } } peers { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } } peers { permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } } }
I20260925 13:11:32.329035 23643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:32.331770 23676 sys_catalog.cc:455] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b11533fa00c2407ea1754b8a93dabe80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } } peers { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } } peers { permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } } }
I20260925 13:11:32.331853 23676 sys_catalog.cc:458] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:32.332324 23643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b11533fa00c2407ea1754b8a93dabe80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } } peers { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } } peers { permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } } }
I20260925 13:11:32.332396 23643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:32.332664 23679 mvcc.cc:205] Tried to move back new op lower bound from 7333240390969479168 to 7333240390961946624. Current Snapshot: MvccSnapshot[applied={T|T < 7333240390969479168}]
I20260925 13:11:32.332722 23680 mvcc.cc:205] Tried to move back new op lower bound from 7333240390969479168 to 7333240390961946624. Current Snapshot: MvccSnapshot[applied={T|T < 7333240390969479168}]
I20260925 13:11:32.334349 23678 catalog_manager.cc:1347] Generated new cluster ID: 6f97ee65eba74f25b9735513faa3fa94
I20260925 13:11:32.334401 23678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 13:11:32.347052 23562 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } attempt: 3
I20260925 13:11:32.348168 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.324042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47422 (local address 127.0.206.126:45443)
0925 13:11:32.324077 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.324079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.330912 (+ 6833us) server_negotiation.cc:407] Connection header received
0925 13:11:32.330940 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.330941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.330952 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.331008 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.339354 (+ 8346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.340073 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.341170 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.341464 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.341503 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.344045 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:32.344046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:32.344047 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:32.344053 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:32.345662 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.346321 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.346322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.346323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.346382 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.346695 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.346696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.346696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.346919 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:32.347025 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.347995 (+ 970us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.348076 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:11:32.350749 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.320407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42334 (local address 127.0.206.125:41537)
0925 13:11:32.320440 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.320442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.330877 (+ 10435us) server_negotiation.cc:407] Connection header received
0925 13:11:32.330976 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.330977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.330986 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.331015 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.331779 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.332539 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.333800 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.334210 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.334261 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.347606 (+ 13345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:32.347607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:32.347608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:32.347614 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:32.349213 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.349946 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.349947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.349947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.350000 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.350179 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.350180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.350180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.350304 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:32.350369 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.350562 (+ 193us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.350635 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 13:11:32.352275 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.331051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55590 (local address 127.0.206.124:37153)
0925 13:11:32.331084 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.331086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.339429 (+ 8343us) server_negotiation.cc:407] Connection header received
0925 13:11:32.339506 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.339506 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.339516 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.339541 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.340140 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.340823 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.341729 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.342005 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.342037 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.348887 (+ 6850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:32.348888 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:32.348889 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:32.348895 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:32.350509 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.351103 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.351104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.351104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.351175 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.351420 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.351420 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.351421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.351553 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:32.351653 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.352120 (+ 467us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.352192 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:11:32.354807 23678 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 13:11:32.355475 23678 catalog_manager.cc:1504] Loading token signing keys...
Sep 25 13:11:32 dist-test-slave-chkf krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790341892, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260925 13:11:32.363435 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.347279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:37153 (local address 127.0.0.1:55598)
0925 13:11:32.347353 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0925 13:11:32.347362 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 13:11:32.347395 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 13:11:32.353042 (+ 5647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 13:11:32.353042 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 13:11:32.353208 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0925 13:11:32.353248 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:32.353250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.354857 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.354857 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:32.355638 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 13:11:32.355640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.356510 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.356511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 13:11:32.356578 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.356967 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 13:11:32.356971 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 13:11:32.358483 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 13:11:32.362075 (+ 3592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:32.362076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:32.362076 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:32.362286 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:32.362619 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 13:11:32.362620 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 13:11:32.362620 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 13:11:32.362676 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 13:11:32.363161 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 13:11:32.363161 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 13:11:32.363203 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 13:11:32.363259 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 13:11:32.363345 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260925 13:11:32.364253 23685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.347465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55598 (local address 127.0.206.124:37153)
0925 13:11:32.349671 (+ 2206us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.349672 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.349680 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:11:32.349787 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.349787 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.349811 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.353356 (+ 3545us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.353433 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.354689 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.355845 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.356375 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.356735 (+ 360us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.358637 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 13:11:32.358639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 13:11:32.358639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 13:11:32.358646 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 13:11:32.361962 (+ 3316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.362384 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.362415 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.362416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.362510 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 13:11:32.362823 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 13:11:32.362824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 13:11:32.362825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 13:11:32.363043 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 13:11:32.363846 (+ 803us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.363924 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.364066 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1819,"thread_start_us":72,"threads_started":1}
I20260925 13:11:32.364473 23678 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80: Generated new TSK 0
I20260925 13:11:32.364632 23678 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 13:11:32.365965 23562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6: Bootstrap starting.
I20260925 13:11:32.366627 23562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6: Neither blocks nor log segments found. Creating new log.
I20260925 13:11:32.368530 23562 log.cc:824] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6: Log is configured to *not* fsync() on all Append() calls
I20260925 13:11:32.369256 23562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6: No bootstrap required, opened a new log
I20260925 13:11:32.373977 23562 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } } peers { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } } peers { permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } }
I20260925 13:11:32.374111 23562 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2ef96ab77204786949428aa4c670ff6, State: Initialized, Role: FOLLOWER
I20260925 13:11:32.374190 23562 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } } peers { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } } peers { permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } }
I20260925 13:11:32.374470 23562 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 13:11:32.374547 23535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 13:11:32.374773 23536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b11533fa00c2407ea1754b8a93dabe80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c2ef96ab77204786949428aa4c670ff6"
I20260925 13:11:32.375785 23535 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6 [term 1 FOLLOWER]: Refusing update from remote peer b11533fa00c2407ea1754b8a93dabe80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 13:11:32.376075 23676 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 13:11:32.376650 23536 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b11533fa00c2407ea1754b8a93dabe80 in term 1.
I20260925 13:11:32.378772 23700 mvcc.cc:205] Tried to move back new op lower bound from 7333240391121592320 to 7333240390961946624. Current Snapshot: MvccSnapshot[applied={T|T < 7333240390969479168}]
I20260925 13:11:32.383018 23706 catalog_manager.cc:1259] Loaded cluster ID: 6f97ee65eba74f25b9735513faa3fa94
I20260925 13:11:32.383065 23706 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6: loading cluster ID for follower catalog manager: success
I20260925 13:11:32.384187 23706 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6: acquiring CA information for follower catalog manager: success
I20260925 13:11:32.384505 23706 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 13:11:32.386821 23688 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b11533fa00c2407ea1754b8a93dabe80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } } peers { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } } peers { permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } } }
I20260925 13:11:32.386927 23688 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:32.386978 23688 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b11533fa00c2407ea1754b8a93dabe80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } } peers { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } } peers { permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } } }
I20260925 13:11:32.387037 23688 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 13:11:32.386857 23702 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b11533fa00c2407ea1754b8a93dabe80. Latest consensus state: current_term: 1 leader_uuid: "b11533fa00c2407ea1754b8a93dabe80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } } peers { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } } peers { permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } } }
I20260925 13:11:32.387086 23702 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2ef96ab77204786949428aa4c670ff6 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 13:11:32.390966 23490 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:32.389488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47438 (local address 127.0.206.126:45443)
0925 13:11:32.389527 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.389529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.390357 (+ 828us) server_negotiation.cc:407] Connection header received
0925 13:11:32.390458 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.390458 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.390468 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.390497 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.390909 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47438: BlockingRecv error: recv got EOF from 127.0.0.1:47438 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 13:11:32.391886 23685 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:32.390105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55614 (local address 127.0.206.124:37153)
0925 13:11:32.390138 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.390139 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.391267 (+ 1128us) server_negotiation.cc:407] Connection header received
0925 13:11:32.391401 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.391402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.391412 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.391440 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.391830 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55614: BlockingRecv error: recv got EOF from 127.0.0.1:55614 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 13:11:32.392313 23566 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:32.389800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42344 (local address 127.0.206.125:41537)
0925 13:11:32.389834 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.389836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.391147 (+ 1311us) server_negotiation.cc:407] Connection header received
0925 13:11:32.391497 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.391498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.391507 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.391535 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.392266 (+ 731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42344: BlockingRecv error: recv got EOF from 127.0.0.1:42344 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:11:32.406153 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.398962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42360 (local address 127.0.206.125:41537)
0925 13:11:32.399002 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.399004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.402900 (+ 3896us) server_negotiation.cc:407] Connection header received
0925 13:11:32.403007 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.403008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.403040 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.403079 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:32.403289 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.404105 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.405241 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.405533 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.405571 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.405799 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:32.405912 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:32.405959 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.406069 (+ 110us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.406086 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 13:11:32.409241 23685 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:32.408515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55624 (local address 127.0.206.124:37153)
0925 13:11:32.408552 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.408554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.408739 (+ 185us) server_negotiation.cc:407] Connection header received
0925 13:11:32.408842 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.408843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.408859 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.408890 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.409194 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55624: BlockingRecv error: recv got EOF from 127.0.0.1:55624 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 13:11:32.414407 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.411387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42368 (local address 127.0.206.125:41537)
0925 13:11:32.411418 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.411420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.411630 (+ 210us) server_negotiation.cc:407] Connection header received
0925 13:11:32.411716 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.411717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.411738 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.411764 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:32.411929 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.412619 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.413668 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.413963 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.414000 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.414180 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:32.414208 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:32.414235 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.414335 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.414351 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 13:11:32.417196 23685 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:32.416495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55634 (local address 127.0.206.124:37153)
0925 13:11:32.416530 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.416531 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.416710 (+ 179us) server_negotiation.cc:407] Connection header received
0925 13:11:32.416816 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.416817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.416827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.416855 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.417146 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55634: BlockingRecv error: recv got EOF from 127.0.0.1:55634 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:11:32.423158 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.420179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42370 (local address 127.0.206.125:41537)
0925 13:11:32.420212 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.420214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.420434 (+ 220us) server_negotiation.cc:407] Connection header received
0925 13:11:32.420523 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.420524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.420532 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.420558 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:32.420715 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.421386 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.422388 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.422644 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.422683 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.422924 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:32.422952 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:32.422979 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.423082 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.423100 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 13:11:32.425932 23685 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:32.425212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55646 (local address 127.0.206.124:37153)
0925 13:11:32.425246 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.425247 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.425437 (+ 190us) server_negotiation.cc:407] Connection header received
0925 13:11:32.425532 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.425533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.425543 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.425573 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.425887 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55646: BlockingRecv error: recv got EOF from 127.0.0.1:55646 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 13:11:32.434203 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.430979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42382 (local address 127.0.206.125:41537)
0925 13:11:32.431011 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.431013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.431236 (+ 223us) server_negotiation.cc:407] Connection header received
0925 13:11:32.431327 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.431328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.431337 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.431363 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:32.431520 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.432198 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.433242 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.433519 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.433560 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.433962 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:32.433992 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:32.434020 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.434128 (+ 108us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.434145 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 13:11:32.437289 23685 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:32.436372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55656 (local address 127.0.206.124:37153)
0925 13:11:32.436408 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.436410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.436589 (+ 179us) server_negotiation.cc:407] Connection header received
0925 13:11:32.436697 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.436698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.436708 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.436913 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.437246 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55656: BlockingRecv error: recv got EOF from 127.0.0.1:55656 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 13:11:32.449764 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.446366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42386 (local address 127.0.206.125:41537)
0925 13:11:32.446400 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.446402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.446649 (+ 247us) server_negotiation.cc:407] Connection header received
0925 13:11:32.446848 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.446849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.446859 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.446888 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:32.447059 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.447769 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.448946 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.449251 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.449294 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.449520 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:32.449551 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:32.449579 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.449685 (+ 106us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.449702 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 13:11:32.452723 23685 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:32.451927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55660 (local address 127.0.206.124:37153)
0925 13:11:32.451989 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.451991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.452231 (+ 240us) server_negotiation.cc:407] Connection header received
0925 13:11:32.452329 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.452330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.452340 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.452369 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.452674 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55660: BlockingRecv error: recv got EOF from 127.0.0.1:55660 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 13:11:32.473086 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.469876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42392 (local address 127.0.206.125:41537)
0925 13:11:32.469914 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.469916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.470180 (+ 264us) server_negotiation.cc:407] Connection header received
0925 13:11:32.470279 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.470280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.470290 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.470317 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:32.470489 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.471245 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.472306 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.472567 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.472611 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.472814 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:32.472859 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:32.472888 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.472995 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.473015 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 13:11:32.476176 23685 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:32.475378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55670 (local address 127.0.206.124:37153)
0925 13:11:32.475416 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.475418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.475630 (+ 212us) server_negotiation.cc:407] Connection header received
0925 13:11:32.475729 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.475730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.475742 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.475771 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.476129 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55670: BlockingRecv error: recv got EOF from 127.0.0.1:55670 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 13:11:32.512300 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.509507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42398 (local address 127.0.206.125:41537)
0925 13:11:32.509558 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.509561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.509704 (+ 143us) server_negotiation.cc:407] Connection header received
0925 13:11:32.509736 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.509736 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.509749 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.509808 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:32.509939 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.510686 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.511551 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.511861 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.511934 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.512057 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:32.512091 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:32.512146 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.512192 (+ 46us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.512211 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260925 13:11:32.514842 23685 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:32.514202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55672 (local address 127.0.206.124:37153)
0925 13:11:32.514258 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.514260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.514294 (+ 34us) server_negotiation.cc:407] Connection header received
0925 13:11:32.514369 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.514370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.514380 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.514452 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.514784 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55672: BlockingRecv error: recv got EOF from 127.0.0.1:55672 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 13:11:32.582352 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.579881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42410 (local address 127.0.206.125:41537)
0925 13:11:32.579959 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.579962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.580064 (+ 102us) server_negotiation.cc:407] Connection header received
0925 13:11:32.580174 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.580175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.580191 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.580248 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:32.580400 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.580895 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.581658 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.581853 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.581939 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.582094 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:32.582128 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:32.582187 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.582242 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.582260 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260925 13:11:32.584769 23685 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:32.584046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55686 (local address 127.0.206.124:37153)
0925 13:11:32.584114 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.584121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.584129 (+ 8us) server_negotiation.cc:407] Connection header received
0925 13:11:32.584176 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.584177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.584186 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.584232 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.584703 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55686: BlockingRecv error: recv got EOF from 127.0.0.1:55686 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 13:11:32.716226 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.713736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42420 (local address 127.0.206.125:41537)
0925 13:11:32.713811 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.713814 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.713925 (+ 111us) server_negotiation.cc:407] Connection header received
0925 13:11:32.714011 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.714011 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.714022 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.714070 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:32.714216 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.714727 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.715482 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.715707 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.715787 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.715937 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:32.715978 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:32.716040 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.716098 (+ 58us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.716117 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260925 13:11:32.718533 23685 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:32.717908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55696 (local address 127.0.206.124:37153)
0925 13:11:32.717966 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.717968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.717995 (+ 27us) server_negotiation.cc:407] Connection header received
0925 13:11:32.718073 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.718074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.718084 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.718129 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.718449 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55696: BlockingRecv error: recv got EOF from 127.0.0.1:55696 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 13:11:32.928644 23676 sys_catalog.cc:455] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b11533fa00c2407ea1754b8a93dabe80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11533fa00c2407ea1754b8a93dabe80" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45443 } } peers { permanent_uuid: "c2ef96ab77204786949428aa4c670ff6" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 41537 } } peers { permanent_uuid: "22cff55758cf49c4ac202c1edc1d4c0d" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 37153 } } }
I20260925 13:11:32.928794 23676 sys_catalog.cc:458] T 00000000000000000000000000000000 P b11533fa00c2407ea1754b8a93dabe80 [sys.catalog]: This master's current role is: LEADER
I20260925 13:11:32.977794 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:32.975684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42428 (local address 127.0.206.125:41537)
0925 13:11:32.975727 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.975729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.975801 (+ 72us) server_negotiation.cc:407] Connection header received
0925 13:11:32.975853 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.975854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.975862 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.975902 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:32.976033 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.976488 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.977196 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:32.977368 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:32.977416 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:32.977541 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:32.977565 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:32.977603 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:32.977728 (+ 125us) server_negotiation.cc:299] Negotiation successful
0925 13:11:32.977743 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260925 13:11:32.980522 23685 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 13:11:32.979815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55698 (local address 127.0.206.124:37153)
0925 13:11:32.979912 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:32.979914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:32.979968 (+ 54us) server_negotiation.cc:407] Connection header received
0925 13:11:32.980039 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:32.980040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:32.980055 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:32.980124 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 13:11:32.980469 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55698: BlockingRecv error: recv got EOF from 127.0.0.1:55698 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 13:11:33.298274 23654 catalog_manager.cc:1259] Loaded cluster ID: 6f97ee65eba74f25b9735513faa3fa94
I20260925 13:11:33.298343 23654 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d: loading cluster ID for follower catalog manager: success
I20260925 13:11:33.299151 23654 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d: acquiring CA information for follower catalog manager: success
I20260925 13:11:33.299467 23654 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 22cff55758cf49c4ac202c1edc1d4c0d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 13:11:33.496086 23823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:33.493449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42434 (local address 127.0.206.125:41537)
0925 13:11:33.493569 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:33.493571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:33.493662 (+ 91us) server_negotiation.cc:407] Connection header received
0925 13:11:33.493762 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:33.493763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:33.493777 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:33.493861 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:33.493992 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:33.494490 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:33.495328 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:33.495544 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:33.495613 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:33.495804 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:33.495842 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:33.495906 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:33.495966 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 13:11:33.495988 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":56,"threads_started":1}
I20260925 13:11:33.500366 23829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:33.497892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55708 (local address 127.0.206.124:37153)
0925 13:11:33.497990 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:33.497992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:33.498041 (+ 49us) server_negotiation.cc:407] Connection header received
0925 13:11:33.498066 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:33.498067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:33.498133 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:33.498184 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:33.498271 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:33.498749 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:33.499436 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:33.499643 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:33.499706 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:33.500011 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:33.500118 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:33.500189 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:33.500225 (+ 36us) server_negotiation.cc:299] Negotiation successful
0925 13:11:33.500244 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":56,"threads_started":1}
I20260925 13:11:33.504974 23823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:33.502077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42438 (local address 127.0.206.125:41537)
0925 13:11:33.502118 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:33.502120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:33.502401 (+ 281us) server_negotiation.cc:407] Connection header received
0925 13:11:33.502495 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:33.502496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:33.502508 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:33.502556 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:33.502679 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:33.503380 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:33.504278 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:33.504546 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:33.504604 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:33.504732 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:33.504762 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:33.504807 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:33.504889 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 13:11:33.504908 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 13:11:33.505005 23836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:33.501996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47450 (local address 127.0.206.126:45443)
0925 13:11:33.502162 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:33.502164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:33.502194 (+ 30us) server_negotiation.cc:407] Connection header received
0925 13:11:33.502227 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:33.502228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:33.502261 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:33.502344 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:33.502459 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:33.503268 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:33.504082 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:33.504328 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:33.504389 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:33.504559 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:33.504686 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:33.504753 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:33.504899 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 13:11:33.504926 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":47,"threads_started":1}
I20260925 13:11:33.505307 23829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 13:11:33.501938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.0.206.124:37153)
0925 13:11:33.501976 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 13:11:33.501978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 13:11:33.502269 (+ 291us) server_negotiation.cc:407] Connection header received
0925 13:11:33.502324 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 13:11:33.502325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 13:11:33.502337 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 13:11:33.502547 (+ 210us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 13:11:33.502580 (+ 33us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:33.503105 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:33.504428 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 13:11:33.504738 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 13:11:33.504809 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 13:11:33.504952 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 13:11:33.504984 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 13:11:33.505033 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 13:11:33.505209 (+ 176us) server_negotiation.cc:299] Negotiation successful
0925 13:11:33.505238 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 13:11:33.506754 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 23419
I20260925 13:11:33.512439 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 23494
I20260925 13:11:33.517189 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskG_Ts_H/build/release/bin/kudu with pid 23569
2026-09-25T13:11:33Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2799 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5605 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (266273 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST